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FOREWORD  AND  ACKNOWLEDGEMENTS 


The  preface  does  as  well  as  perhaps  any  remarks  in  introducing  these 
proceedings.  -'The  theoretical  operability  deficit  (e.g.,  for  aircraft)  is 
modulable  through  the  Incorporation  of  such  advances  as  cockpit  engineering 
decisJon  augmentation  technology,  task  allocation  methods,  voice  interactive 
technology,  etc.  What  advances  are  we  to  rely  on,  however,  for  deflating  the 
growing  lag  in  maintainability?  The  reports  contained  here  represent  a 
compendium  of  current  thinking  on  the  matter.  They  were  produced  by  a  diverse 
body  of  subject  matter  experts,  and  they  comprise  quite  an  array  of 
philosophies.  The  common  denominator  Is,  of  course,  the  human  factor./ 

Although  this  document  is  one  of  only  a  very  few  dedicated  exclusively  to 
human  factors  in  maintenance,  it  is  no  doubt  the  most  recent.  The 
Information,  innovation,  and  more  than  anything,  the  enthusiasm,  that  were  so 
vigorously  and  successfully  exchanged  at  the  conference  will  represent  only 
the  first  such  endeavor;  the  Department  of  Defense  Human  Factors  Engineering 
Technical  Advisory  subGroup  for  Human  Factors  in  Logistics  (LOGSTAG)  has 
eagerly  sought  to  sponsor  the  symposium  on  a  yearly  basis. 

The  conference  and  this  publication  would  not  have  been  what  they  were 
without  the  tenacious  dedication  of  several  key  players:  Ms.  Louida  Murray, 
Ms.  Laura  Hitchcock,  Dr.  Joe  Lambert,  and  Ms.  Sharon  Morgan.  Their  untiring 
capacity  for  work,  exemplary  professionalism,  and  enthusiastic  support  were 
truly  inspirational. 

The  comments,  opinions,  and  philosophies  contained  in  or  inferred  from  the 
reports  that  follow  are  not  necessarily  those  of  the  U.S.  Navy,  nor  of  any  of 
the  respective  sponsors,  unless  otherwise  stated. 


Dennis  K.  McBride,  Ph.D. 

LT,  MSC,  USNR 

Manager,  Design-for-Maintalners,  and 
Chairman,  Design-for-Maintalners  Conference,  1982 
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A  Hunan  Factors  Design-for-Maintainers 
Technology  Development  Program 


Dennis  K.  McBride  Joseph  V.  Lambert 

Naval  Air  Development  Center  Eagle  Technology,  Inc. 

Warminster,  Pennsylvania  Warminster,  Pennsylvania 


The  problem  of  maintaining  airborne  systems  has  grown  considerably  over 
the  past  two  decades.  With  the  Increased  size  and  ever-growing  complexity  of 
modern  aviation  weapon  systems,  estimates  for  maintaining  them  now  range  from 
1/4  to  1/3  of  the  entire,  yearly  DOD  budget'.  Furthermore,  it  has  been 
estimated  that  as  many  as  1/3  of  all  military  personnel  are  detailed 
exclusively  to  maintenance  and  support  functions.  Apparently,  however, 
traditional  problem  solutions  have  not  worked.  A  typical  Navy  squadron  today 
may,  for  example,  have  only  about  50  percent  of  Its  aircraft  available  for 
full  operational  use.  An  analysis  of  the  3-M  data  on  the  F-14,  to  isolate  one 
problem  area,  reveals  (1)  that  because  of  excessive  mean  elapsed  maintenance 
times  (EMT),  a  $2.3  B  excess  Inventory  of  F-14s  Is  needed  in  order  to  maintain 
a  prescribed,  mission-capable  force,  and  (2)  that  maintainer  errors  alone 
(e.g.,  diagnostic  false  alarms  or  maintenance-produced  damage)  account  for  a 
staggering  share  of  unscheduled  maintenance  costs  (Fuchs  4  Inaba,  1979).  This 
means  that  an  additional  1.23  malntalners  are  needed  per  aircraft,  merely  to 
recover  from  performance  errors'. 

Clearly,  when  It  comes  to  operational  readiness,  reliability  is,  and 
always  has  been  a  key  issue  (see,  for  example,  Willoughby,  l98l),  but  because 
of  the  acceleration  In  the  subsystem  complexities  which  characterize  modern 
aircraft,  even  If  overall  system  reliability  could  manage  to  sustain  present 
day  levels  (or  even  Improve,  miraculously),  measured  maintainability  would  no 
doubt  continue  to  decline.  While  such  technological  advances  as  Built-in-Test 
(BIT)  and  modularization  certainly  show  promise,  unfortunately,  there  Is  good 
reason  to  believe  that  early  predictions  for  their  success  were  quite  probably 
overly  optimistic  (e.g.,  BIT  reliability  for  the  F-14  Is,  at  best, 
disappointing).  Polling  of  the  participants  at  this  symposium  (over  376 
man-years  pooled  experience)  reflects  this  disappointment  (Appendix  A; 

McBride,  1982).  BIT's  perceived  rellabllty  as  a  cure  for  diagnostic  Ills,  Is 
for  this  diverse  body  of  symposlasts,  at  least,  lukewarm.  And  its  perceived 
future  potential  Is  only  slightly  better.  Furthermore,  forecasts  for  the 
availability  (dwindling  supply)  and  trainabillty  (accelerating  costs)  of  a 
future,  qualified  population  of  organizational -level  malntalners  are 
pessimistic. 

The  essence  of  maintainability  Is,  of  course,  fast,  safe,  efficient 
repair;  and  at  Its  heart  are  people-related  variables— factors  such  as 


diagnostic  behavior,  decision/cognitive  complexities,  reading  comprehension, 
job  aiding  technology,  accessibility,  psychonotor  coordination,  anthropometric 
matchups,  transfer  of  training,  man-to-machine  information  transfer,  and 
systematization.  These  human  factors  must  be  comprehensively  and 
systematically  examined,  and  the  performance-based  discoveries  which  their 
research  provides  must  be  elegantly  intermeshed  as  design  criteria,  standards, 
and  specifications.  Design-for-Maintainers  (DFM)  is  NAVAIRSYSCOM1 s  (340-F) 
multldlscipllned  technology  development  program,  aimed  specifically  at  this 
most  crucial  manpower/ readiness  problem.  The  RAD  effort  is  performed  by  the 
Maval  Air  Development  Center  (NADC  Human  Factors  Technology  Development, 
Aircraft  and  Crew  Systems  Technology  Development  Directorate,  Warminster, 
Pennsylvania)  under  Program  Elements  62757N  and  63710N.  {The  Office  of  Naval 
Research  sponsors  a  related  and  coordinated  effort.) 

Following  is  what  must  be  only  a  superficial  overview  of  some  of  the 
philosophical  issues  which  underlie  the  steering  of  DFM  technology 
development.  In  order,  the  following  topics  are  addressed:  (1)  Reliability 
and  Maintainability;  (2)  The  Human  as  a  Factor;  and,  as  an  introduction  to 
the  reports  following,  a  brief  introduction  to  (3 )  Design-for-Maintainers. 


Reliability  and  Maintainability 


Blanchard  and  Lowery  (1969),  tell  us  that  maintainability  (M)1,  as  an 
"accredited"  engineering  discipline,  evolved  as  a  product  of  the  many 
reliability  (R)  engineering  efforts  of  the  1940s  and  1950s.  Perhaps  as 
significant  as  any  M-related  historical  landmark,  the  Pentagon,  in  1954, 
formally  ($)  acknowledged  and  adopted  M  as  the  curative  counterpart  to  R,  the 
preventive.  Since  that  time,  both  R  and  M  have  experienced  the  ad  hoc 
attention  so  typically  devoted  to  adoptees,  the  "Oh  yeh,  let's  not  overlook  R 
4  M"  policy.  Although  R  and  M  comprise  (as  subsumed  under  logistics) 
something  called,  perhaps  simplistlcally,  but  at  least  singularly,  "Systems 
Availability  Technology,"  it  is  a  curious  but  common  perception  that  these 
disciplines  have  evolved  as  mutual  adversaries.  Competition  for  resources  ($) 
no  doubt  shares  some  responsibility  for  their  evolutionary  branching;  but  for 
whatever  reasons,  "Big  R/Little  M,"  "Big  R/Important  M,"  etc.,  although  aimed 
at  precisely  the  same  goal— availability,  and  exploiting  precisely  the  same 
approach— system  reliability  (machine,  man),  continue  to  regard  the  other  as 
threat. 


Regardless,  one  serious  question  for  both  R  and  M  philosophers  has  to  do 
with  what  seems  somehow  counterintuitive  to  a  surprisingly  large  camp.  That 
is,  although  component/subsystem  reliability  has  continued  to  increase  over 
recent  years,  system  reliability  has  continued  to  dwindle.  The  solution  is, 
of  course,  that  the  number  of  components  which  comprise  modern  aircraft  has 
also  increased  over  the  years,  and  since  overall  system  reliability  is  a 
product  of  the  combined  "subreliabilities,"  availability  has  declined.  To 
risk  insulting  the  sophisticated  reader,  take  the  following  example.  Suppose 
the  mean  reliability  for  essential  components  for  a  particular  system  (i.e.,  a 
defect  means  aircraft  is  grounded)  experiences  a  dramatic,  linear  growth  from 
say  .9  to  .99  over  some  ten-year  period.  For  the  same  period,  let's  say  that 


1  M  *  concept/discipline,  ff  =  measure  of  M 


the  number  of  assemblies,  parts,  etc.— the  components  which  constitute  the 
aircraft— and  the  number  of  component  Interfaces  also  Increase  linearly,  by  a 
theoretical  factor  of  .1,  so  that  the  "average"  aircraft  at  the  end  of  our 
10-year  stuc(y  window  has  10  percent  more  components  (and  even  more  interfaces) 
than  Its  "average"  predecessor.  Exercising  a  simple  cumulative  binomial  model 
of  failure  prediction  shows  that  there  is  not  an  increase,  but  a  decrease  in 
aircraft  availability.  That  is,  the  probability  that  no  parts  will  be 
portrayed  as  faulty  on  some  theoretical,  time-frozen  snapshot  of  aircraft 
availability  has  actually  decreased.  Furthermore,  these  hypothetical  figures 
represent  rather  conservative  depictions  of  trend;  in  reality,  component 
reliabilty  has  obeyed  an  increasing,  but  negatively-accelerated  growth  curve, 
and  component  proliferation  is  more  typically  positively  accelerated  (or 
sigmoidal).  This  all  means,  of  course,  that  technology must  drive  component 
reliability  upwards  by  a  very  immodest  exponential  factor  in  order  to  maintain 
constancy  in  system  availability.  History  shows  clearly  that  this  requisite 
cannot  be  met. 

For  it  is  written,  then,  that  systems  fail.  The  issue,  therefore,  becomes 
one  of  repair.  And  it  is  the  repairman's  reliability— the  probability  that 
his  performance  error  will  not  render  a  system  unavail  able— that  now  becomes  a 
central  consideration.  So,  how  much  variance  in  TURNAROUND-TIME  do  maintainer 
(vis  a  vis,  supply)  variables  ACTUALLY  account  for?  The  answer  is  not  clear, 
largely  because  of  the  question  of  the  immediacy  of  maintainer  involvement  in 
many  of  the  multitudinous  facets  of  maintenance  activity.  Not  surprisingly, 
however,  this  lack  of  clarity  does  not  yield  a  paucity  of  answers.  The 
disappointingly  few,  though  highly  significant,  contributions  in  the 
literature  (Blanchard  A  Lowery,  1969;  Crawford  A  Altman,  1972;  Rigney,  Cremer, 
A  Towne,  1965;  Topmlller,  1964;  see  Hsu  A  Theisen,  these  Proceedings  for  a 
review)  suggest  strongly  that  the  human  interface  is  in  fact  key.  Fuchs  and 
Inaba  (1979),  for  example,  have  shown  that  maintainer  errors  can  be  reduced  in 
simulation  by  as  much  as  97  percent  with  appropriate  manipulations  of  design. 
Furthermore,  M  practitioners  do  not  point  smartly  to  the  numbers  of  technical 
publications  devoted  to  HFM— there  are  only  a  few.  The  man-hours  invested, 
however,  are  staggering;  and  a  quick  look  at  the  results  of  the  opinion  poll 
(Appendix  A;  McBride,  1982)  leave  no  doubt  that  those  who  worry  about  M 
problems,  scientist  and  maintainer  alike,  share  a  mind:  variance  contributed 
by  the  human  interface  is  legion. 

On  the  other  hand,  many  operational  maintainers  and  officers  perceive  the 
above  account  as  a  tenuous  indictment  of  technician  skill  level,  and  instead, 
invoke  supply  (viz.,  parts  availability)  as  the  chief,  if  not  the  only  nemisis 
of  efficient  turnaround.  There  are  literally  scores  of  supply-related 
management  problems  for  which  human  factors  technology  has  been,  and  should 
be,  providing  solutions.  As  one  real  example  of  this  potential,  voice 
technology  has  shown  considerable  promise  for  the  remediation  of 
catalog/lnventory/dlstributlon/dlsposal  problems.  For  present  purposes, 
however,  attention  is  devoted  to  the  design  of  equipment  and  its  interaction 
with  human  variables. 

The  Human  as  a  Factor 

Maintainer  reliability  is  underpinned,  of  course,  by  a  number  of  factors, 
the  first  of  which,  could  be  characterized  as  demographic  or  personnel.  In 
the  Navy,  for  example,  there  is  a  generally  reliable,  three-way  classification 


of  first-tour  Navy  maintainers.  Comprising  the  first  category  are  those  who 
need  a  job  tomorrow;  they  impulsively  apply  for  enlistment,  pass  the  necessary 
screening  exercises,  and  by  a  "thought-out"  Navy  selection  process,  they  are 
assigned  to  a  track  which,  in  many  cases,  will  ultimately  become  a  severely 
truncated  career.  These  individuals  have  an  excessive  desertion  rate,  and 
their  performance  is  generally  well  below  average— observations  interpretable 
as  motivational  problems.  The  second  category,  those  who  are  career- (read 
security)  minded,  typically  come  from  disadvantaged  socioeconomic  backgrounds 
and  are  typically  poor  in  such  abilities  as  reading,  communication, 
comprehension,  etc.  The  third  grouping,  fortunately,  represents  the  largest 
population  t>60%)  of  entering  Navy  maintainers.  They  are  bright,  eager  to 
learn,  suffer  fewer  motivational  problems,  and  perform  exceptionally  well. 
Unfortunately,  over  90  percent  of  these  maintainers  leave  for  better 
pay/conditions  after  their  first  tour.  The  problem  for  the  Navy,  of  course, 
is  the  resulting  negative  relationship  between  time- in-service  (and  thus, 
assigned  responsibility  level)  and  basic  skill  level.  Of  those  who  do  not 
choose  to  make  the  Navy  a  career,  the  probability  that  a  maintainer  will 
terminate  his  Navy  service  increases  with  measured  ability  level  and 
experience.  In  other  words,  those  who  cost  most  to  train  and  do  the  best  job 
don' t  stay. 

The  second  conglomeration  of  human  issues  which  help  predict  maintainer 
performance  can  be  classified  as  training  and  educational  factors.  The  Navy 
has  an  elaborate  assignment  scheme  which  factors  in  such  variables  as 
recruiting  commitments,  PCS  costs,  school  quotas,  etc.  for  determining 
training  and  specialization  tracks  for  sailors.  Theory,  applications,  and 
on-site  training  are  intermeshed  according  to  any  number  of  variables,  and 
performance  is  evaluated  continually.  Scores  of  arguments  have  been  and  can 
be  raised  with  regard  to  the  effectiveness  of  current  training  methodology 
(e.g.,  see  sections  in  these  Proceedings)  but  one  limiting  factor  surely 
underwrites  the  ultimate  impact  of  all  the  "mega  dollars"  of  training  and 
simulator  expenditures.  That  factor,  as  outlined  in  the  previous  paragraph, 
asserts  that  the  effectiveness  of  any  training  or  selection  Innovation  is  no 
better  than  the  a  posteriori  1  i kel i'hood  that  selectees  and  trainees  continue 
in  service.  Unfortunately,  psychologists  have  long  known  that  those  who 
profit  most  from  traditional  training  are  not  universally  those  who  begin 
training  with  the  lowest  skill  levels. 

Furthermore,  as  the  sophistication  of  aircraft  has  grown,  the  nature  of 
the  underlying  variables  which  govern  maintainer  performance  has  also 
changed.  Figure  1  illustrates  the  point.  Forty  years  ago,  it  was  conceivable 
(and  imminently  practicable  with  the  draft)  to  recruit  (select)  a  body  of 
maintai/iers,  who  with  a  modicum  of  "repairman"  experience  and  a  minimum  of 
training,  comprised  a  qualified  population  of  operational  maintainers.  As 
aircraft  complexity  increased,  however,  more  and  more  training  was  found  to  be 
necessary,  so  that  in  the  1980s,  training  expenditures  (Including  simulator 
technology)  for  maintenance  and  logistics  have,  by  necessity,  soared. 

Summarized  rather  simpllstically,  when  the  effectiveness  of  personnel  and 
training  methodologies  begin  to  approach  their  technological  end  points— 
whether  because  of  accelerating  system  sophistication,  technician  skill 
deficits,  or  whatever— it  becomes  painfully  clear  that  in  many  ways  it  is  less 
expensive  to  manipulate  the  design  of  equipment  than  to  "manipulate  the  design 
of  human  capability  or  its  expression. 


Figure  1.  Schematic  portrayal  of  the  Impact  of  increased  aircraft 

sophistication  on  the  variables  underlying  maintainer  performance. 

Design-for-Maintainers 

DFM  is  based  philosophically  on  the  tenets  outlined  above.  Presented 
simply,  the  program  can  be  characterized  as  in  Figure  2.  The  ultimate 
concern,  of  course,  is  the  useful  output  provided  now  and  in  the  future  by 
DFM.  This  output  comes  in  two  mutually  interactive  varieties:  (1)  engineer¬ 
ing  solutions  to  existing  problems,  as  for  example,  the  F-14  ECP  depicted  in 
Figure  3,  and  (2)  technological  advances  for  the  prevention  of  future 
problems.  The  methodology  which  provides  these  products  is  common  to  both, 
and  it  involves,  first  of  all,  the  application  of  state  of  the  art  HFM 
principles  toward  well -recognized,  M  problem  targets.  This  "application"  may 
take  several  approaches.  For  example,  a  known,  incorporated  design  feature  or 
change  may  have  been  driven,  let's  say,  by  the  need  for  greater  accessibility 
to  a  particular  avionics  component.  Aviation  3-M  data,  or  perhaps  squadron 
reports,  are  analyzed  after  the  fact  to  determine  the  ft  consequences  of  the 
engineering  feature.  The  application  may,  however,  not  be  "after-the-fact," 
but  instead,  it  might  be  an  attempt  to  verify  in  a  controlled  fleet  setting 
(e.g.,  NAMTRADET)  or  well -control led  laboratory,  the  predicted  impact  of  a 
single  or  set  of  HFM  features  on  fT. 

The  critical  task  of  validation  then  begins.  Here,  the  effectiveness  of 
the  design/redesign  Inputs  is  scrutinized;  the  nonproductive  elements  are 
discarded  or  rethought,  and  the  productive  ones  are  retained.  A  factor- 
analytlc-like  approach  then  partitions  the  valid  design  enhancements  into  such 
traditional  categories  as  accessibility,  diagnostic  complexity,  biodynamic 
stress,  or  It  suggests  new  ones.  As  the  validation  process  continues,  the 
methodology  Is  expanded  to  other  components,  subsystems,  and  systems,  where 
validation  and  update  continue  to  support  this  progression. 


DATA  SOURCES 

EXISTING  CREATED 

•  3  -  M 

•  SQUADRON  REPORTS 
•  FLAG  TftF 

•  NAVSAFCEN 

•  PLRFORV.ANCF 


ENGINEERING 

applications  transfer 

FOR  IMMEDIATE  SOLUTIONS 


•  ECP 

•  WSIP 

•  Df  SIGN  CONSULT 


DFM  METHODOLOGY 

STATE  OF  THE  ART  - - 

I - -  VALIDATE 

1 - -  iipananr 


WORK? 


TECHNOLOGY 

DESIGN  TO  IMPACT  ALL 
FUTURE  ACQUISITIONS 

•  st  andards/spe  cif  (Cations 

•  CHECKLISTS 

•  CAO'CAM.  MAM 

•  TRADEOFF  EVALUATIONS 


Figure  2.  Outline  of  Design-for-Maintainers  program. 

RECOMMENDED  CHANGE: 

PROVIDE  AN  ADAPTER  FOR  EACH  SENSING  ELEMENT  NO.  1 
TO  AVOID  ACCESS  REQUIREMENT  EXTERNAL 
TO  THE  ENGINE  NACELLE. 


ESTIMATED  BENEFIT: 
SAVE  2  TO  3  HOURS  IN 
REMOVAL/INSTALLATION 


Figure  3.  One  example  of  an  ECP  which  would  impact  maintainer  efficiency. 
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Table  1 


SELECTED  HFM  MAJOR  INSTALLATION  RECOMMENDATIONS 
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An  example  of  an  ongoing  effort  might  serve  well  here.  When  the  Navy's 
F-18  became  a  blueprint  reality,  HFM  engineers  made,  literally,  hundreds  of 
inputs  aimed  at  improving  the  aircraft's  maintainability.  Whereas  the  F-14 
had  shown  an  MMH/FH  figure  in  excess  of  40  (i.e.,  a  man-week  effort  required 
to  prepare  the  aircraft  for  the  next  1  hour  of  flight'.),  these  F-18  design 
features  (Table  1)  were  intended  to  bring  that  figure  down  to  around  11,  a 
much  more  tolerable  maintenance  burden.  With  the  establishment  of  the  first 
F-18  squadron,  the  success  of  those  state  of  the  art,  engineerng  features  can 
now  be  tracked  via  a  systematic  examination  of  aviation  3-M  data.  And  the 
analysis  of  these  successes  and  unproductive  solutions  underlies  the 
development  of  ongoing  HFM  technology.  If,  for  example,  certain  design 
features  consistently  induce  diagnostic  false  alarms/false  removals,  and 
others  lead  more  typically,  say,  to  equipment  damage  errors,  then  standards, 
specifications,  checklists,  even  CAD/CAM  interactive  models  can  be  generated 
to  decrease  the  prevalence  of  their  incorporation  into  the  design  of  future 
systems.  Such  associations  are  being  borne-out  currently  by  DFM.  These 
preventive  measures— technological  design  aids— have  proved  in  the  past,  and 
can  continue,  to  save  millions  in  maintenance  man-hours  and  dollars. 

Currently,  and  as  many  of  the  following  reports  show,  DFM  methodology  is 
exploiting  several  data  bases,  but  it  is  deliberately  concentrating  on  a 
relatively  small  cross  section  of  platforms.  The  F-14,  because  of  its 
recognized  F  problem,  its  mature-yet-relatively-new  status  as  a  fighter 
aircraft,  the  vast  number  of  ECPs  associated,  and  its  ample  maintenance  data 
reserve,  represents  quite  a  target  for  HF  attention.  Other  aircraft  are  also 
prime  attractions:  the  EC-130  (unique  0-level  maintenance  scheme),  the  SH-2F 
(a  rotary  wing),  the  F-18,  and  the  F-4.  Plans  are  in  work  for  the  VT-X.  The 
data  sources  also  vary  from  3-M,  to  FLAG  TAE,  squadron  reports,  structured 
interview,  structured  maintenance  activity  analysis,  and  on  to  NAMTRADET 
performance  records. 

The  program  is  solution-driven.  Yet,  neither  engineering  curatives,  nor 
technological  preventives  will  represent  a  M  panacea.  The  effectiveness  of 
the  approach  is  no  better  than  the  extent  to  which  the  user  community— the 
PMAs,  advanced  concepts  technologists,  safety  engineers,  class  desks,  RAM 
engineers,  operational  maintainers,  and  squadron  commanders— are  actually 
intermeshed  as  active  DFM  subscribers.  Transitional  funding  and  sponsorship, 
therefore,  must  be  two-dimensional:  (1)  DFM  has  in  fact  succeeded  in 
establishing  a  relatively  secure  6.1  (with  ONR),  6.2,  and  now  6.3  transitional 
base.  Growth  on  this  dimension  must  be  invigorated,  however,  with  (2)  an  even 
more  convincing  and  sustaining  growth  on  the  axis  of  "user  acceptance."  DFM 
^beginning  to  make  this  claim. 
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A  Current  Application  of  Design-for-Maintainers  (DFM)  Technology 


Frank  H.  Fuchs 

XYZYX  Information  Corporation 
Canoga  Park,  California 


An  application  of  DFM  technology  is  taking  place  right  now  in  the  city  of 
Detroit.  We  have  there  a  nonnilitary  project  involving  maintenance  on  a  bus. 
The  bus  in  question  is  the  GMC  RTS  II  (Figure  1).  The  RTS  II  became 
operational  in  1976  and  is  now  in  use  in  nearly  every  major  city  in  the 
country. 

In  the  project  described  here,  DFM  technology  covers  the  following: 
information  packages  for  use  by  maintenance  personnel,  solutions  to  problems 
posed  hy  inadequate  system  design,  inappropriate  support  equipment, 
counterproductive  maintenance  practices,  and  measurement  of  bus  fleet 
performance  reflecting  maintenance  effectiveness. 

Our  mission  in  Detroit  is  to  help  the  Department  of  Transportation  improve 
the  availability  of  the  RTS  II  through  more  efficient  maintenance.  Our 
original  strategy  was  to  drive  down  the  mechanic  error  rate  by  means  of  Job 
Performance  Aids  (JPAs).  Hov/ever,  as  we  moved  into  the  work  environment,  we 
discovered  other  problems  that  also  needed  attention.  Some  of  those  problems 
are  described  in  this  report. 

The  project  referred  to  covered  three  systems  on  the  RTS  II.  In  this 
particular  account,  we  will  concentrate  on  only  one  of  them:  the  Heating  and 
Air  Conditioning  System.  First,  we  will  look  at  the  system  in  question;  this 
will  provide  a  context  for  the  materials  to  follow.  Next,  we  will  see  some 
samples  of  the  JPAs  and  other  Information  products  delivered.  We  will  then 
examine  some  maintenance  problems  caused  by  both  the  system  designer  and  the 
maintainers  themselves.  We  will  describe  our  experiences  in  trying  to 
implement  our  program.  Finally,  we  will  explain  our  approach  to  program 
evaluation. 


•  Refrigerant  circulation 
t  Water  circulation 

•  Air  circulation 

•  Control s 

In  certain  ways,  each  subsystem  is  similar  to  Its  counterpart  on  other 
buses.  Mapy  of  the  components  are  identical,  as  are  individual  functions. 

What  makes  this  system  unique  is  that,  for  the  first  time,  the  functions  of 
heating  and  cooling  are  brought  together  as  a  single  system  rather  than  two 
related  systems.  Another  unique  feature  is  the  control  subsystem.  Mode 
selection  is  accomplished  by  thermostats  sensitive  to  external  air 
temperature.  As  we  will  see  later,  this  feature  is  not  necessarily  beneficial 
to  the  mechanic. 

Views  of  the  respective  subsystems  as  shown  in  Figures  3,  4,  5,  and  6  are 
meant  to  show  that  the  various  components  are  in  fact  familiar  to  the 
maintenance  community.  That  is  a  valuable  characteristic  in  any  new  system. 
Another  point  of  importance  to  the  mechanic  is  component  location.  Here 
again,  the  designer  did  a  good  job.  Physical  access  is  not  a  big  problem  on 
this  system. 

Information  Products  Provided 


Now  that  we  have  been  introduced  to  the  system  itself,  we  are  ready  to  see 
the  first  element  of  assistance  given  to  the  mechanics  through  our 
program— the  information  products.  Chief  among  these  products  are  the  JPAs 
(Figure  7). 

The  list  of  JPAs  reflects  the  entire  scope  of  maintenance  jobs  applicable 
to  the  system.  Most  of  the  jobs  entail  component  replacement  and 
service-related  actions  at  the  component  level.  A  small  number  of  jobs,  such 
as  troubleshoot  and  check,  apply  at  the  system  and  subsystem  levels.  This 
distribution  is  typical  of  most  systems. 

Figures  8  and  9  represent  sample  pages  from  two  JPAs  and  are  shown  in 
order  to  convey  the  flavor  of  the  information  involved.  One  illustrates  a 
replacement  job,  the  other,  check  and  service.  Note  the  following  features  of 
Figure  8: 

§  Job  location  and  hardware  appearance  shown  pictorial ly 

•  Numbered  arrows  marking  each  point  to  be  touched 

t  Action  statements  carrying  those  same  numbers 

•  Short,  simple  statements 

Figure  9  features  include: 

t  Specific  criteria  where  a  decision  must  be  made 

•  Clear  routing  out  of  decision  point 


All  JPAs  are  formatted  in  this  manner. 


REPLACE  BOOSTER  PUMP 


Remove  Booster  Pump  (continued) 


8.  Remove  vortex  screen  (4)  from  inlet 
hose  (3).  See  if  there  is  damage 
on  screen. 

In  the  following  step,  hold  BOOSTER 

PUMP  while  removing  flange  bolt6  (1). 

9.  Remove  four  flange  bolts  (1)  from 
pump  clamp  (2). 

10.  Remove  BOOSTER  PUMP  from  coach. 


END  OF  ACTIVITY 


Figure  8.  Sample  JPA. 


Check  For  Air  la  Sygtea  (continued) 


.  See  If  reading  on  high  pressure 
gauge  (2)  Is  within  three  psig  of 
pressure  on  Temperature 
Relationship  Table,  page  9. 

If  reading  Is  within  three  psl  of 
value  on  table,  go  to  Step  17, 

Page  9. 

If  reading  is  not  within  three  pslg 
of  pressure  on  table,  continue. 


gauge  (2). 


To  obtain  proper  temperature  reading, 
thermometer  or  sensor  must  be  fastened 
to  condenser  line  with  thermo-mastic 
tape  4 

9.  Place  thermometer  on  condenser 
line  (1).  Wrap  with  tape.  Vait 
ten  minutes,  then  note 
temperature . 

10.  Using  two  temperature  readings, 
determine  average  temperature. 

11.  Note  reading  on  high  pressure 


Certain  tasks  are  common  to  several  jobs.  In  order  to  control  against 
repetition,  such  tasks  are  best  covered  outside  the  JPAs  and  then  simply 
referred  to  as  the  need  arises.  In  this  project,  the  document  used  to  do  that 
is  called  a  Skill  Aid.  Eleven  Skill  Aids  were  required  for  the  Heating  and 
Air  Conditioning  System  (see  Table  1). 

Strictly  speaking,  the  term  "skill"  may  not  be  the  most  accurate  one  to 
use.  We  frequently  think  of  skill  in  connection  with  sensorimotor  faculties. 
Rut,  in  maintenance,  those  kinds  of  abilities  are  seldom  needed.  Maintenance 
jobs  are  much  more  likely  to  be  driven  by  the  need  for  information. 

Figure  10  is  a  sample  page  from  a  Skill  Aid.  The  presentation  techniques 
employed  are  similar  to  those  used  in  JPAs.  The  halide  torch,  by  the  way,  is 
a  device  used  to  check  for  leaks  in  refrigerant  lines.  When  the  flame  comes 
into  contact  with  leaking  Freon,  the  flame  changes  color. 

The  final  information  product  to  be  exhibited  (Figure  11)  is  the  system 
explanation.  The  purpose  of  a  system  explanation  is  to  give  the  mechanic  an 
understanding  of  how  the  parts  of  a  system  work  together  to  produce  the 
required  outputs.  Such  information  is  vital  to  effective  troubleshooting. 

The  page  shown  in  Figure  11  is  from  the  first  section  of  the  explanation. 
The  picture  will  be  recognized  as  one  illustrated  earlier.  The  first  section 
is  expressed  in  very  simple  terms.  The  intent  being  merely  to  establish  a 
foundation  for  the  detailed  passages  that  follow.  The  system  explanation, 
like  all  the  other  information  products  developed  for  the  project,  makes  use 
of  well-founded  principles  of  presentation  and  learning. 

Maintenance  Problems 

The  Heating  and  Air  Conditioning  system  on  this  particular  model/series 
produced  a  number  of  interesting  maintenance  problems.  Some  were  caused  by 
the  system  designer,  others  by  the  maintainer.  It  is  sometimes  hard  to  tell 
where  one  stops  and  the  other  begins. 

Refrigerant  leaks  are  a  case  in  point.  This  bus  has  had  a  persistent 
problem  with  leaks.  Some  experts  say  it  was  due  primarily  to  vibration. 

Others  say  the  chief  cause  was  overpressurization.  We  don't  know  which  side 
is  right.  We  do  know,  however,  that  the  leaks  have  been  there,  and  that  the 
maintainers  were  not,  at  first,  dealing  with  them  effectively. 

Leaks,  by  the  way,  are  a  triple-threat  problem  in  refrigeration  systems. 
First,  of  course,  they  allow  the  Freon  to  escape  gradually,  thus  reducing  the 
cooling  power  of  the  system.  Second,  they  permit  the  entry  of  air  bearing 
moisture  which  combines  with  the  Freon,  hydrochloric  acid  is  formed,  which 
causes  Internal  corrosion,  and  further  aggravation  of  the  original  leakage 
problem.  Finally,  the  leaks  make  necessary  a  higher  frequency  of  corrective 
actions  which  happen  to  be  very  time-consuming  in  nature. 

With  these  facts  in  mind,  we  can  now  look  at  two  maintenance  practices 
that  were  clearly  counterproductive.  The  first  practice  has  to  do  with 
cleaning  the  outside  of  the  bus.  Implicated  are  both  cleaning  personnel  and 
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Introduction 

The  halide  torch,  also  called 
"sniffer”,  is  used  to  detect  and  locate 
leaks  In  the  refrigerant  system.  The 
torch  consists  of  the  following  parts: 

o  fuel  tank  (6) 
o  valve  (5) 

o  burner  (2)  with  copper  plate  (A) 
o  pick-up  hose  (1) 

The  torch  may  burn  propane,  alcohol  or 
acetylene . 

When  the  valve  (5)  is  opened,  fuel 
flows  from  the  tank  into  the 
burner  (2).  The  fuel  should  be  ignited 
within  a  few  seconds  of  opening  the 
valve.  Otherwise  too  much  fuel  escapes 


into  the  surrounding  r .  Too  much 
fuel  in  and  around  the  burner  can 
result  in  a  burned  hand  the  fuel 

is  ignited,  or  even  causr  a  fire  or 
explosion  hazard.  Once  th  flame  (3) 
is  burning,  the  valve  is  u»eu  to  adjust 
the  size  of  the  flame. 

The  heat  generated  by  the  flame  draws 
air  through  the  pick-up  hose  (1)  into 
the  burner.  The  color  of  the  flame 
indicates  the  presence  or  absence  of 
Freon  gas  in  the  air.  Therefore,  the 
color  of  the  flame  should  be  observed 
when  air  without  Freon  is  drawn  into 
the  burner.  This  "normal"  color 
depends  on  the  type  and  quality  of  the 
fuel  being  burned. 


HALIDE 

TORCH 


Figure  10.  Sample  page  from  a  skill  aid. 
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The  control  subsystem  uses  pneumatic  and 
electrical  devices  such  as  valves,  sole¬ 
noids  and  switches.  Control  "decisions” 
are  made  by  the  thermostats. 

Many  of  the  automatic  control  components 
are  located  in  the  heating  and  air  condi¬ 
tioning  control  box  that  is  mounted  on 


the  rear  of  the  evaporator  housing  in  the 
heating  and  air  conditioning  compartment. 

Manual  controls  are  located  on  the  driv¬ 
er's  control  panel.  Key  electrical  con¬ 
trol  components  are  located  in  the  rear 
apparatus  box  in  the  engine  compartment 
and  in  the  evaporator  housing. 


Figure  11.  System  explanation 


the  system  designer.  The  designer  led  off  by  placing  the  refrigerant 
condenser  in  the  engine  compartment,  right  beside  the  radiator.  Remember  now, 
that  the  radiator  has  beside  it,  a  fan  that  pulls  outside  air  in  through  the 
radiator  for  the  purpose  of  cooling  the  engine  (Figure  12). 

In  this  case,  the  fan  acts  like  a  vacuum  cleaner.  Each  day,  dirt  and 
debris  are  drawn  in  from  the  street  and  deposited  on  the  condenser  fins.  The 
initial  result  is  to  restrict  the  flow  of  air  through  the  condenser,  thus 
reducing  the  condenser's  ability  to  transfer  heat  from  the  refrigerant  to  the 
outside  air.  At  this  point,  cleaning  personnel  take  over.  In  their  zeal  to 
get  the  condenser  back  to  normal,  they  use  the  strongest  measure  available  to 

them:  steam  cleaning.  The  steam  is  effective  in  cleaning  the  condenser 

fins.  The  problem  is,  it  is  also  effective  in  overheating  the  refrigerant 

lines.  When  refrigerant  is  overheated,  it  builds  up  pressure  rapidly.  For 
example,  refrigerant  at  120  degrees  Fahrenheit  exerts  160  pounds  of  pressure. 
At  136  degrees,  the  pressure  increases  to  200  pounds.  By  150  degrees,  the 
pressure  escalates  to  800  pounds.  The  refrigerant  lines,  meanwhile,  are 
designed  for  250  pounds  of  pressure.  Thus,  the  high  pressures,  created  by  the 
excessive  heat  of  the  steam,  place  a  severe  strain  on  the  lines,  greatly 
increasing  the  chances  of  further  leaking.  Fortunately,  the  practice  of  steam 
cleaning  around  the  refrigerant  lines  was  stopped  after  attention  was  drawn  to 
its  had  effects. 

The  second  example  of  a  counterproductive  maintenance  practice  involves 
support  equipment  (Figure  13).  Here  again,  the  refrigerant  lines  play  an 
important  role.  Leaks  must  be  dealt  with  as  follows: 

1.  Fmpty  system  of  refrigerant 

2.  Repair  leaks  by  soldering 

3.  Evacuate  system  to  remove  moisture 

4.  Recharge  system  with  refrigerant 

The  key  step  in  the  process  is  evacuating  the  system  to  remove  moisture.  As 
indicated  earlier,  moisture  in  a  refrigerant  system  creates  big  problems.  The 
system  may  be  emptied  for  repair  by  means  of  a  vacuum  pump  and  a  standard 
gauge  set. 

To  remove  moisture  from  the  system,  however,  the  same  equipment  is  not 
adequate.  The  vacuum  pump  used  for  normal  purposes  will  take  far  too  long  to 
do  the  job  of  removing  moisture.  A  larger  pump  is  needed.  The  standard  gauge 
set,  meanwhile,  measures  in  Inches  of  mercury,  whereas  the  evacuation  level 
required  calls  for  measurement  in  terms  of  microns.  The  criterion  value  is 
450  microns.  One  inch  of  mercury  equates  to  25,400  microns.  Coordination 
with  the  maintenance  superintendent  remedied  this  problem.  Each  evacuation 
station  was  equipped  with  a  heavier  pump  and  a  gauge  set  capable  of  measuring 
in  microns.  The  new  measuring  device  is  called  a  thermistor  gauge  set. 

The  next  two  examples  of  maintenance  problems  lead  to  difficulties  In 
troubleshooting.  Both  were  created  by  the  designer.  The  first  involves  water 
circulation  and  associated  controls. 

As  we  said  earlier,  the  water  circulation  subsystem  (and  associated 
controls)  is  responsible  for  coach  heating.  Here  is  what  happens  when  the 
coach  is  in  the  heating  mode  (Figure  14):  a  thermostat  in  the  control  group 


THERMISTOR  GAUGE  SET 


Figure  13.  Improper  maintenance  support  equipment 


senses  the  temperature  of  the  air  In  the  coach.  If  the  air  is  too  cool, 
pneumatic  control  signals  are  sent  out,  activating  the  heating  components. 

The  key  heating  component  Is  the  water  modulation  valve  which  controls  the 
flow  of  hot  water  from  the  engine  cooling  system  through  the  heaters.  If  the 
water  modulation  valve  and  other  components  are  working  right,  warm  air  is 
blown  Into  the  Interior  of  the  coach.  Eventually,  the  temperature  of  the 
coach  interior  Is  again  sensed  by  the  thermostat  and  the  cycle  is  repeated. 

The  process  Involves  ten  major  components,  plus  pneumatic  lines,  water 
lines,  electrical  wires  and  connecting  hardware.  The  problem  with  the 
existing  arrangement  Is  that,  when  the  system  fails,  it  Is  very  difficult  to 
determine  which  component  Is  responsible.  The  design  forms  a  closed  loop 
unbroken  by  any  Indicators  or  controls.  The  only  way  to  approach  a 
malfunction  is  to  test  each  item  individually. 

Project  analysts  responded  to  the  problem  by  constructing  a  special  test 
fixture  (Figure  15).  The  fixture  contains  two  controls,  a  pressure  gauge,  a 
hose,  and  connector  hardware.  The  connector  hardware  allows  the  fixture  to  be 
Installed  In  the  existing  pneumatic  control  line  (Figure  16)  running  into  the 
water  modulation  valve.  Along  with  this  fixture  Is  a  thermometer,  placed 
Inside  the  coach.  With  this  new  setup,  the  troubleshooter  can  run  a  variety 
of  tests,  all  from  one  convenient  location. 

Figure  17  shows  how  the  test  fixture  would  be  represented  In  our  original 
diagram.  Note  the  control  and  the  Indicator  of  the  fixture  and  the 
thermometer  In  the  coach.  Consider  now,  two  quick  examples  of  the  types  of 
tests  possible. 

In  the  first  example,  the  troubl eshooter  can  use  the  start  signal  to  the 
heating  components.  By  checking  the  temperature  of  the  air  flow  at  the 
outlets  in  the  coach,  he  can  quickly  determine  whether  or  not  the  heating 
components  are  working.  If  they  are,  then  the  trouble  must  reside  in  the 
control  components. 

The  second  example  applies  If  the  heating  components  are  working.  The 
test  fixture  allows  the  troubleshooter  to  vary  the  amount  of  heat  delivered  by 
the  heating  components.  The  temperature  variations  are  picked  up  by  the 
thermometer  Inside  the  coach.  For  each  temperature  valve,  a  corresponding 
pressure  valve  should  be  registered  on  the  test  fixture  indicator.  Different 
kinds  of  discrepancies  In  the  temperature/pressure  relationship  point  to 
different  groups  of  components  as  possible  causes  of  the  malfunction.  The 
special  test  fixture  has  become  a  popular  piece  of  hardware.  The  maintenance 
superintendent  has  taken  steps  to  make  It  available  to  all  garages. 

The  second  example  of  troubleshooting  difficulty  Involves  the  blower  motor 
control  circuits.  There,  all  components  are  interconnected  In  such  a  way  that 
the  failure  of  any  one  could  shut  down  the  entire  group.  The  blower  motors 
are.  In  fact,  wired  In  series  (Figure  18). 

Ideas  for  redesign  were  easily  generated.  One  In  particular  Is  shown  In 
Figure  IQ.  Note  that  It  consists  merely  of  converting  the  series  circuit  into 
two  parallel  circuits.  The  advantages  to  the  troubleshooter  are  obvious. 


re  17.  Coach  in  heating  mode  under  test 
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Figure  19.  Blower  motor  control  circuit  (suggested  revision) 


Unfortunately,  it  was  not  possible  to  get  this  change  authorized.  The 
inefficient  original  design,  thus,  has  had  to  be  dealt  with  by  careful 
construction  of  the  Troubleshoot  JPA. 


Program  Implementation 


While  the  information  products  were  being  developed,  a  great  deal  of 
analysis  was  taking  place.  This  analysis  led  to  the  discovery  of  the 
erroneous  maintenance  practices.  Correction  of  those  practices  was  made 
possible  by  a  cooperative  maintenance  superintendent. 


Mechanics  were  taken  through  basic  training  on  the  Heating  and  Air 
Conditioning  System.  Because  of  the  power  of  the  JPA  concept,  basic  training 
was  accomplished  in  less  than  24  "hours"  per  class.  JPAs  were  then  placed  in 
all  the  garages,  at  points  where  the  mechanics  could  gain  access  to  them 
easily. 


To  expand  the  opportunity  for  newly-trained  mechanics  to  work  on  the 
system,  a  preventive  maintenance  campaign  was  established.  It  calls  for 
direct  use  of  key  JPAs  in  the  work.  It  is  staffed  entirely  by  newly-trai ned 
mechanics. 


A  quality  control  plan  was  adopted,  calling  for  the  work  of  the  preventive 
maintenance  teams  to  be  checked  for  their  first  few  experiences.  A  feedback 
loop  was  established  whereby  supervisors  responsible  for  the  preventive 
maintenance  teams  could  be  kept  Informed  of  their  work  quality.  Two  means 
were  provided.  One  is  the  record  generated  by  the  quality  control  effort. 

The  other  is  a  record  showing  repeat  jobs  and  the  mechanics  associated  with 
them. 


Program  Evaluation 


A  special  effort  has  been  undertaken  to  obtain  a  practical  evaluation  of 
the  Detroit  program.  This  effort  has  Involved  quantitative  measures  applied 
both  before  and  after  training.  One  measure  Is  based  on  road  calls.  The 
other  is  based  on  drive-in  complaints.  Both  represent  unscheduled  maintenance 
actions  of  the  type  Detroit  would  like  to  avoid,  whenever  possible. 
Fortunately,  the  normal  recordkeeping  system  at  Detroit  requires  the 
documentation  of  each  maintenance  task  action.  Key  data  items  recorded  at 
that  time  are  (Figure  20): 


•  Date  of  occurrence 

•  Coach  number 

•  System  Involved 

•  Mechanic  Involved 


With  these  elements,  It  is  possible  to  construct  reports  that  reflect 
performance  and  maintenance  effectiveness.  Preventive  maintenance 
transactions  are  recorded  separately. 

Some  preliminary  analyses  have  already  been  made,  focusing  on  repeat 
maintenance  actions  (Figures  21  and  22).  Such  repeats  are  taken  to* reflect 
the  presence  of  mechanic  errors.  We  realize,  of  course,  that  all  repeats  are 
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Figure  20.  Records  identifying  unscheduled  maintenance 
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Figure  22.  Repeat  maintenance  incidents  due  to  error  (heating  and  air 
conditioning  system  -  summer  and  winter  months). 


not  the  result  of  errors.  Some  are  due  to  normal  equipment  failure.  Even  so, 
as  shown  here,  the  actual  number  of  Incidents  Is  considerably  greater  than  the 
number  expected.  The  difference  between  the  two  represents  the  potential  for 
Improvement.  We  now  believe  that  there  are  several  paths  to  that 
Improvement.  One  is  better  Information  for  the  mechanic.  Another  is  better 
maintenance  practices.  Another  is  better  equipment  design,  especially  as  it 
affects  troubleshooting. 

With  specific  regard  to  program  evaluation,  we  are  organizing  fleet 
performance  data  as  shown  in  Tables  2  and  3.  That  Is,  we  are  counting 
Incidents,  not  repeats.  And,  we  are  referencing  them  against  miles  driven. 

We  have  isolated  the  systems  covered  by  the  program.  For  each  system,  we 
are  totaling  road  calls  and  drive-in  complaints  per  month.  This  will  allow  us 
to  establish  a  baseline  record  denoting  performance  prior  to  training.  Data 
collection  will  be  continued  for  a  year  after  training.  It  will  then  be 
possible  to  compare  performance  in  corresponding  months,  from  one  year  to  the 
next. 

Conclusion 

The  project  at  Detroit  has  reached  the  point  where  the  mechanics  trained 
on  the  Heating  and  Air  Conditioning  System  are  about  to  start  working  in  their 
respective  garages.  As  indicated  earlier,  all  known  erroneous  maintenance 
practices  have  been  corrected  and  all  recommended  new  support  equipment  has 
been  obtained.  A  preventive  maintenance  campaign  is  scheduled  to  start  on  15 
March,  1982.  Observers  will  be  watching  closely.  We  will  know  from  the 
quantitative  data  to  what  extent  we  have  succeeded. 
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Jesse  Orlansky 
Joseph  String 


Institute  for  Defense  Analyses 
Alexandria,  Virginia 


There  are  several  reasons  to  be  Interested  in  how  well  maintenance 
personnel  perform  their  job.  The  major  one  is  that  their  performance 
influences  the  operational  readiness  of  weapon  systems  in  the  field.  Another 
is  that  knowledge  about  the  quality  of  job  performance  is  needed  to  evaluate 
the  effectiveness  of  methods  used  to  select  and  train  maintenance  personnel. 
Surprisingly,  little  objective  data  are  available  to  document  how  well 
technicians  do  what  they  are  supposed  to  do.  Up  to  now,  methods  of  selection 
and  training  have  been  validated  almost  entirely  on  the  basis  of  how  well 
maintenance  technicians  perform  at  school  rather  than  on  the  job. 

Measures  of  Job  Performance 

The  following  types  of  measures  appear  valid  for  describing  how  well 
maintenance  personnel  perform  on  the  job: 

Number  of  malfunctions  diagnosed  correctly 

Average  amount  of  time  required  to  diagnose  correctly  various  types  of 
malfunctions 

Number  of  replace  and/or  repair  actions  performed  per  unit  time 

Maintenance  man-hours  per  operating  hour 

Operational  (combat)  readiness  of  units  supported  by  maintenance  personnel 

Maintenance  man-hours  per  maintenance  requirement  (action  or  task) 

Number  of  nonfaulty  assemblies  removed  unnecessarily 

Damage  to  equipment  during  corrective  maintenance 

Failure  to  remove  faulty  equipment# 

This  list  of  measures  is  meant  to  be  suggestive  rather  than  complete.  If 
data  for  these  measures  were  to  be  collected,  they  would  obviously  have  to  be 
based  on  complete  records  or,  at  least,  on  a  representative  sample  of 
equipments,  malfunctions,  personnel,  and  working  conditions. 

The  military  services  operate  large  maintenance  management  data  systems 
that  provide  much  detailed  Information  on  the  current  maintenance  status  of 
military  equipment.  These  data  systems,  identified  in  Table  1,  are  discussed 


elsewhere  In  this  volume  (see  String  and  Orlansky,  pp.  59).  These  systems 
were  designed  to  provide  information  needed  to  manage  maintenance  and  logistic 
services  and  not  to  relate  the  performance  of  military  technicians  to  methods 
of  selection  and  training. 

There  appears  to  be  no  routinely  available  source  of  data  that  describes 
the  performance  of  maintenance  personnel  in  any  of  the  military  services.  A 
few  special  investigations  (i.e.,  ad  hoc  efforts)  have  been  reported  and  these 
provide  the  basis  for  the  present  paper.  All  of  these  concern  only  the 
removal  of  nonfaulty  parts  during  corrective  maintenance.  Corrective 
maintenance  at  the  organizational  level  is  limited  to  "on-equipment"  repair  or 
the  removal  of  suspect  assemblies  from  equipment  end-items.  The  removal  of 
nonfaulty  assemblies  would  appear  to  indicate  inadequate  job  performance  by 
the  organizational  maintenance  technician.  Information  that  nonfaulty  parts 
have  been  removed  arises  later  when  intermediate  level  maintenance  personnel 
cannot  find  a  malfunction. 

Data  on  the  removal  of  nonfaulty  parts  are  easy  to  identify  and  can  be 
conveniently  collected.  At  best,  they  can  describe  some,  but  obviously  not 
all,  aspects  of  the  quality  of  job  performance  of  maintenance  personnel.  Some 
qualifications  on  the  interpretation  of  these  data  will  be  discussed  later. 

Table  1 


Maintenance  Management  Data  Systems  Used  by  the  Military  Services 


Maintenance  Management  System 

Service 

Name 

Short  Title 

Army 

The  Army  Maintenance  Management  System 

TAMMS 

Navy 

Naval  Ships'  Maintenance  and  Material 
Management  System 

Ships'  3-M 

Navy 

Naval  Aviation  Maintenance  and 

Material  Management  System 

Aviation  3-M 

Air  Force 

Air  Force  Maintenance  Management 

Systems 

66-1  and  66-5 

Results 


A  summary  of  seven  reports  on  the  removal  of  nonfaulty  parts  during 
corrective  maintenance  appears  in  Table  2.  It  concerns  the  maintenance  of 
aircraft,  armored  vehicles,  and  the  electrical  components  of  other  automotive 
vehicles.  Some  large  data  samples  are  involved,  e.g.,  72  F-14A  aircraft  over 
a  period  of  one  year;  all  maintenance  actions  (a  total  of  0.4  million  actions) 
in  the  Navy  on  four  aircraft  over  a  period  of  one  year;  the  smallest  sample  is 
for  all  maintenance  on  electrical  components  at  an  Army  base  for  one  month. 

The  sources  of  the  data  are  some  of  the  records  collected  routinely  by  the 
maintenance  management  systems  of  the  three  Services. 


Nonfaulty  parts  were  removed  in  4  to  43  percent  of  all  corrective 
maintenance  actions  in  these  data;  the  median  value  of  11  data  sets  is  15 
percent.  The  removal  of  nonfaulty  parts  accounts  for  9  to  32  percent  of  all 
maintenance  man-hours  (for  three  cases  where  such  data  were  reported). 
According  to  one  study,  technicians  fail  to  find  a  faulty  part  or  they  damage 
a  good  part  in  about  10  percent  of  all  corrective  maintenance  actions  (Gold, 
Kleine,  Fuchs,  Ravo,  &  Inaba,  1980). 

These  data  suggest  that  inadequate  performance  by  technicians  contributes 
to  the  "not-ready"  status  of  military  equipment.  Other  factors  would  include 
the  unavailability  of  spare  parts,  test  equipment,  and  up-to-date  technical 
documentation.  For  example.  Gold  et  al.,  (1980)  estimate  that  an  average  of 
22  percent  of  the  F-14A  aircraft  were  not  ready  over  a  one-year  period  for 
reasons  due  to  supply.  According  to  a  questionnaire,  about  50  percent  of  551 
Army  technicians  believed  that  repetitive  maintenance  (same  malfunction)  of 
Army  helicopters  was  due  primarily  to  inadequate  test  equipment,  trouble¬ 
shooting,  and  standard  maintenance  practices;  about  20  percent  gave  inadequate 
training,  tools,  and  maintenance  manuals  as  a  secondary  cause  (Holbert  & 
Newport,  1975).  These  findings  appear  to  identify  a  significant  problem  in 
military  maintenance  but  do  not  suggest  a  means  to  its  solution. 

The  data  sample  is  small  and  may  not  be  representative.  The  removal  of 
nonfaulty  parts  may  not  always  be  an  inappropriate  action,  e.g.,  the  test 
equipment  may  not  be  capable  of  distinguishing  between  a  faulty  and  nonfaulty 
part;  if  the  technician  is  under  pressure  to  have  equipment  ready  for  a 
mission,  he  may  remove  and  replace  a  large  number  of  assemblies  without  tests 
in  order  to  make  sure  that  the  malfunctioning  unit  has  been  removed.  Finally, 
the  data  apply  to  all  maintenance  actions  within  large  units  and  not  to  the 
performance  of  particular  individuals. 

One  particular  value  of  data  describing  the  quality  of  performance  of 
maintenance  personnel  on  jobs  in  operational  settings  would  be  their  use  in 
validating  selection  standards  for  recruiting  and  assigning  personnel  to 
career  paths  and  for  evaluating  the  effectiveness  of  various  methods  of 
training  (e.g.,  conventional  instruction  compared  to  computer-based 
instruction,  use  of  maintenance  training  simulators  as  opposed  to  actual 
equipment  training).  As  a  general  matter,  the  effectiveness  of  military 
selection  and  training  has  been  evaluated  on  the  basis  of  performance  of 
technicians  at  school  and  not  on  the  job.  The  latter  is  the  more  relevant 
criterion. 

Another  possible  use  of  these  data  v/ould  be  to  the  human  factors 
engineering  of  maintenance  support  equipment  and,  perhaps,  of  the  operational 
equipment  as  well.  It  might  be  that  inadequate  human  factors  design  of 
equipment  increases  the  difficulty  of  identifying  and  replacing  failed 
equipment  and  leads,  to  some  extent,  to  the  removal  of  nonfaulty  parts. 

It  is  conceivable  that  the  data  generated  through  maintenance  management 
systems  of  the  military  services  could  be  modified  to  provide  information  on 
the  performance  of  maintenance  technicians.  These  systems  were  designed 
primarily  to  manage  maintenance  operations  and  cannot  be  faulted  for  not 
providing  information  relevant  to  the  performance,  selection  and  training  of 
personnel.  A  prototype  system  for  providing  some  of  this  information  has  been 
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developed  and  is  now  being  tested  by  the  U.S.  Army  Research  Institute  (Katz  & 
Drillings,  1981).  Called  The  Ariny  Maintenance  Performance  System,  it  records 
the  work  experience  (time  on  each  technical  task  in  the  maintenance  battalion) 
and  training  (courses  and  qualification  tests)  of  each  maintenance 
technician.  This  record  system  is  not  meant  to  be  part  of  The  Army 
Maintenance  Management  System.  It  would  be  used  by  work  supervisors  and 
training  managers;  each  soldier  would  carry  his  own  record  of  experience  and 
skill  history.  It  does  not  appear  that  this  record  system  would  contain 
information  about  effective  and  ineffective  performance,  e.g.,  time  to 
diaqnose  malfunctions,  success  and  failure  to  diagnose  malfunctions  of  various 
types. 
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Maintainability,  as  defined  quantitatively,  is  the  probability  that  an 
equipment  can  be  restored  to  operable  conditions  within  a  given  period  of  time 
(Goldman  fi  Slattery,  1977).  In  the  past  two  decades  maintainability  has  posed 
a  serious  problem  for  military  systems.  With  the  advent  of  modern  technology, 
the  complexity  and  the  size  of  equipment  systems  has  imposed  severe  demands  on 
maintainers.  This  excessive  maintenance  workload  has  impaired  operational 
readiness. 

To  overcome  these  maintenance  burdens,  a  variety  of  efforts  have  been 
devoted  toward  enhancing  maintainers'  productivity.  Researchers  have  been 
concentrating  on  exploring  relationships  between  maintainability  and  system 
design  parameters  such  as  equipment  design,  personnel  selection  and  training, 
support  facilities,  and  the  operating  environment.  Some  human  factors 
specialists  try  to  apply  their  knowledge  to  design  equipments  that  are  not 
only  minimizing  maintenance  errors  but  maximizing  maintainers'  capabilities, 
and  to  develop  effective  training  methods  to  enhance  maintainers'  skill 
levels.  Even  though  we  recognize  the  importance  of  personnel  and  logistics 
factors,  we  are  not  dealing  with  these  factors  within  the  scope  of  the 
Pesign-for-Maintainers  Program.  Instead,  we  will  concentrate  on 
maintainability  research  which  predicts  and  evaluates  maintenance  performance 
as  affected  by  equipment  design. 

Maintainability  research  in  equipment  design  serves  three  basic  purposes. 
First,  it  develops  methodologies  for  maintenance  performance  measurement  and 
prediction  (e.g.,  Shriver  &  Foley,  Jr.,  1974).  Secondly,  it  is  used  to 
evaluateThe  impact  of  current  equipment  design  on  maintenance  performance, 
which  in  turn  provides  diagnostic  feedback  to  design  engineers.  In  this 
light,  maintainability  research  can  serve  as  an  evaluation  tool  for  the 
development  of  Engineering  Change  Proposals  (ECPs)  or  other  weapon  improvement 
programs.  Thirdly,  it  is  employed  to  determine  the  absolute  and  relative 
contributions  of  various  design  variables  to  maintenance  performance,  which  in 


between  different  design  features.  In  this  vein,  maintainability  research 
acts  as  a  ground  work  for  the  derivation  of  design  guidelines  for  use  in  the 
early  design  phase  to  improve  the  maintainability  of  future  equipment. 


Approaches  to  maintainability  research  thus  far  can  be  classified  into 
th.ree  categories:  (1)  time-synthesis  methods,  (2)  correlational  methods,  and 
(3)  experimental  methods.  The  purpose  of  this  survey  is  to  describe  and 
analyze  these  available  approaches,  and  to  examine  the  methodological  issues 
germane  to  these  approaches. 

Current  Approaches  to  Maintainability  Research  and  Modeling 

Quite  often,  maintainability  requirements  are  expressed  in  terms  of 
mean-time-to-repair  (MTTR)  and/or  mean-down-time  (MDT)  because  of  the  close 
relationship  between  these  Indices  and  system  effectiveness  measures  such  as 
operational  availability.  A  number  of  maintainability  models  were  developed 
to  predict  maintenance  times  from  system  and  equipment  design,  and  to  provide 
an  indication  of  design  compliance  with  specified  quantitative  system 
requirements.  Generally,  these  prediction  models  can  be  categorized  into  two 
groups:  (1)  time-synthesis  methods,  in  which  a  "bottom-up"  approach  is 
applied.  That  is,  it  starts  by  'defining  a  number  of  component  activities  in  a 
maintenance  task.  The  maintenance  time  distributions  associated  with  these 
component  activities  are  then  synthesized  to  form  the  time  distribution  for 
the  higher-order  maintenance  task.  And  (2)  correlational  methods,  which 
predict  the  maintenance  time  of  a  task  from  scores  on  checklists  which  are 
designed  to  evaluate  system  design  characteristics  such  as  equipment  design 
features,  personnel  requirements,  and  support  facilities.  A  thorough  review 
of  these  prediction  models  has  been  conducted  by  Rigney  and  Bond  (1964),  and 
Smith,  Westland,  and  Crawford  (1970).  Here,  only  a  brief  description  and 
examination  of  methodological  issues  existing  in  these  models  will  be 
discussed. 

Time-Synthesis  Methods 

The  ARINC  model,  procedure  I  in  MIL-HDBK-472,  is  based  on  the 
"building-block"  procedure  and  the  concept  of  transferability.  The 
building-block  procedure  states  that  a  maintenance  task  can  be  broken  down 
into  a  number  of  "elemental  activities,"  i.e.,  simple  maintenance  actions. 

The  time  distribution  of  each  maintenance  category  (e.g.,  preparation,  fault 
location,  etc.)  can  be  synthesized  by  the  addition  of  time  distributions 
associated  with  its  constituent  activities.  These  time  distributions  of 
maintenance  categories,  time  distributions  of  logistics  factors  and 
administrative  factors  and  equipment  component  reliability  figures,  in  turn, 
synthesize  into  system  downtime.  The  synthesis  principle  assumes  that:  (1) 
elementary  activities  in  a  maintenance  category  are  independent  of  system 
design  factors,  while  the  frequency  of  occurrence  of  an  elementary  activity  is 
s  affected  by  system  design  factors.  The  compound  probability  of  conjunctive 

1  occurrence  of  several  constituent  activities  is  determined  by  a  simple 

i  multiplication  of  the  probabilities  of  occurrence  of  these  activities.  In 

!  other  words,  it  implies  that  these  constituent  activities  are  discrete  actions 

;  and  nalntalners  perform  these  activities  in  some  predefined  sequence.  The 

i  transferability  principle  holds  that  data  obtained  from  one  airborne 

j  electronic  and  electro-mechanical  system  can  be  generalized  to  those  similar 

systems  which  are  operated  under  comparable  conditions.  The  design  variables, 


used  as  Input  neasures  In  the  ARINC  model,  are  limited  to  the  number,  type, 
and  location  of  components.  The  estimated  measures  Is  the  time  distribution 
of  a  total  system  downtime.  The  application  of  this  model  is  only  limited  to 
the  final  phase  of  equipment  development. 

Another  illustration  of  time-synthesis  methods  is  the  FEC  model --procedure 
II  in  MIL-HDBK-472.  Although  the  FEC  model  follows  similar  basic  synthesis 
principles,  there  are  several  major  differences  between  the  FEC  model  and  the 
ARINC  model.  One  is  that  the  estimated  maintenance  time  here  is  taken  as  the 
sum  of  average  times  associated  with  maintenance  task  elements.  Another 
difference  is  that  the  FEC  model  includes  more  design  features  and  personnel 
requirements.  The  FEC  model  assumes  that  a  maintenance  task  time  is 
determined  by  the  level  at  which  maintenance  functions  are  performed  and 
diagnostic  and  repair  methods  are  used  to  locate  and  correct  failures  of  each 
part.  The  other  difference  is  that  the  FEC  model  can  also  be  applied  to 
estimate  preventive  maintenance  time. 

There  are  several  objections  to  time- synthesis  methods.  The  synthesis 
principle  does  not  always  hold.  It  is  only  applicable  to  those  maintenance 
tasks  which  are  composed  of  a  series  of  simple,  discrete  actions  performed  in 
a  fixed  sequence.  In  the  real  world,  however,  naintainers  do  not  necessarily 
perform  maintenance  task  components  in  a  predefined  sequence.  Rather,  they 
may  perform  some  activities  such  as  troubleshooting  actions  in  real  time. 
Furthermore,  Information  and  sensorimotor  feedback  resulting  from  a  preceding 
activity  tends  to  affect  either  the  occurrence  or  the  performance  of  the 
following  activity  in  a  way  such  that  the  organization  of  motor  patterns  of  a 
maintenance  action  is  changed  or  an  alternative  strategy  is  adopted. 

Secondly,  It  is  doubtful  that  the  data  Interpretations  associated  with  these 
two  models  can  be  extrapolated  to  future  systems.  Although  both  models  assume 
the  general izability  of  data  obtained  from  one  type  of  electronic  equipment  to 
other,  similar  types,  the  extent  of  data  applicability  depends  on  the 
dimensions  (e.g.,  equipment  functions,  maintenance  concepts,  or  system  design, 
etc.)  by  which  the  systems  are  judged  to  be  similar.  Thirdly,  both  the  FEC 
and  ARINC  models  are  rather  insensitive  to  the  effects  of  design  variables  on 
maintenance  performance.  One  problem  is  that  the  design  variables  dealt  with 
in  these  models  are  limited  to  just  a  few  of  the  physical  characteristics  of  a 
component  or  a  complete  system.  In  applying  human-machine  interface  design 
principles,  the  models  ignore  the  psychological  attributes  underlying  various 
physical  characteristics.  In  other  words,  the  models  fail  to  specify  how 
these  physical  characteristics  Impose  excessive  stress  on  naintainers.  For 
example,  does  the  large  number  of  replaceable  components,  which  is  dealt  with 
in  the  ARINC  model,  specify  the  high  degree  of  complexity  of  a  maintenance 
task  or  other  psychological  factors?  In  fact,  physical  characteristics  have  a 
more  profound  effect  on  maintenance  performance  than  the  models  assume. 

Unless  these  psychological  variables  which  underlie  physical  characteristics 
are  known,  it  is  very  difficult  to  predict  maintenance  performance  from 
physical  characteristics.  A  second  major  problem  within  the  models  is  that 
the  dependent  measures  estimated  are  contaminated  criteria  of 
maintainability.  That  is,  a  high  system  downtime  or  mean-active-maintenance 
time  is  a  combined  index  of  poor  reliability  as  well  as  poor  maintainability. 
Since  poor  design  presumably  exerts  its  major  effect  on  maintenance  time, 
these  two  specific  time  measures  are  not,  in  their  present  form,  appropriate 
indicators  of  the  effects  of  poor  design  on  maintenance.  Thus,  the  time 


measures  In  these  models  do  not  provide  designers  with  Information  concerning 
how  to  Improve  maintenance  performance. 


Correlational  Methods 


In  the  RCA  model,  procedure  III  in  MIL-HDBK-472,  it  is  postulated  that  the 
duration  of  system  downtime  is  a  function  of  such  system  design  parameters  as 
equipment  configuration  characteristics,  support  facilities,  and  personnel 
requirements.  Therefore,  system  downtime  can  be  estimated  by  inserting  scores 
obtained  from  three  design-related  checklists  into  a  linear  regression 
equation.  Moreover,  the  evaluation  of  these  checklists  Is  dependent  upon  the 
results  of  a  step-by-step  maintenance  task  analysis. 

Although  portions  of  the  RCA  prediction  procedure  can  be  used  to  evaluate 
the  relative  maintainability  of  alternative  designs,  this  model  does  not  allow 
design  engineers  to  perform  trade-offs  between  various  design  variables 
because  the  relative  importance  of  all  the  design  features  in  the  checklist 
are  assumed  to  be  equal.  Another  problem  in  this  model  is  related  to  the 
general izabllity  of  the  regression  equation  from  one  equipment  to  another. 

The  same  regression  equation  has  been  employed  to  predict  maintainability  of 
different  equipment  types.  However,  validity  studies  need  to  be  performed 
across  different  types  of  equipment  in  order  to  assure  this  general izability. 
The  third  problem  concerns  the  subjectivity  involved  in  checklist  evaluation. 
The  validity  of  checklist  evaluation  relies  both  on  the  availability  of 
detailed  information  concerning  design  features,  and  on  the  knowledge  of  the 
scoring  technique  and  engineering  principles  an  evaluator  has.  Therefore,  the 
prediction  technique  is  doomed  to  fail  if  an  appropriate  maintenance  analysis 
is  not  performed  or  training  in  engineering  and  psychometric  principles  is  not 
available  to  an  evaluator. 

To  remedy  the  foregoing  weaknesses  of  the  RCA  model,  a  series  of 
correlational  studies  were  later  done  by  Lintz  and  his  collegues  (Lintz,  Loy, 
Brock,  A  Potempa,  1973;  Lintz,  Loy,  Hopper,  &  Potempa,  1973;  Potempa,  Lintz  & 
Luckew,  1975).  One  of  the  goals  of  their  studies  was  to  demonstrate  that  the 
multiple  regression  approach  can  be  a  viable  method  for  serving  as  an 
objective  estimation  of  maintenance  performance  from  design  characteristics. 

In  order  to  investigate  the  impact  of  design  features  extensively  and  to 
establish  a  comprehensive  data  base,  an  inclusive  list  of  design  features  was 
generated  from  a  wide  range  of  avionics  equipments  and  22  design  variables 
were  later  selected  on  the  basis  of  the  ratings  of  their  relative  importance. 
Factor  analyses  and  correlational  analyses  were  conducted  to  evaluate  the 
relationships  between  design  features  and  maintenance  time,  and  errors  on 
checkout  procedures  of  ten  avionics  equipments.  Prediction  equations  were 
thus  derived.  With  regard  to  organizational  level  maintenance  performance, 
the  findings  showed  that  performance  time  can  be  predicted  from  a  combination 
ot  design  features  such  as  the  number  of  controls  and  displays,  the 
reliability  of  test  equipment,  and  the'  percentage  of  checkout  to  lowest 
line-replaceable-units  (LRUs).  The  probability  of  committing  a  maintenance 
error  can  be  predicted  from  another  set  of  design  features  such  as  the 
accessibility  of  components,  the  percentage  of  plug-in-circuits,  the 
percentage  of  connectors  which  can  be  incorrectly  connected,  the  number  of 
special  conditions  required  for  checkout,  complexity  of  test  equipment 
operation,  and  percentage  of  checkout  to  lowest  LRUs. 


The  high  correlations  between  design  features  and  maintenance  performance 
found  in  these  studies  suggest  that  the  multiple  regression  approach  can  be  a 
powerful  method  for  predicting  maintenance  performance  from  an  evaluation  of 
design  features.  Moreover,  these  findings  constitute  a  valuable  source  of 
hypotheses  for  further  research.  Investigators  should  be  aware,  however,  of 
several  weaknesses  inherent  in  multiple  regression  analysis.  One  weakness  is 
that  a  regression  analysis  only  tells  design  engineers  what  design  features 
affect  maintenance  performance.  Questions  as  to  how  design  features  influence 
maintenance  performance  still  remain  unanswered.  Therefore,  the  regression 
method  does  not  lead  directly  to  firm  design  specifications.  The  second 
weakness  of  multiple  regression  is  that  the  reliability  of  regression  weights 
decreases  when  a  large  sample  size  is  not  available  and  the  number  of 
independent  variables  is  relatively  large.  Even  though  some  attempts  have 
been  made  to  reduce  the  number  of  independent  variables  by  performing  a  factor 
analysis  beforehand,  it  is  very  difficult  for  design  engineers  to  use  those 
"factors,"  derived  from  the  factor  analysis,  as  guidelines  in  designing 
equipment.  Another  weakness  of  multiple  regression  is  that  it  is  very 
difficult  to  determine  the  relative  contributions  of  various  independent 
variables  when  those  independent  variables  are  intercorrelated. 

Unfortunately,  that  is  the  case  in  maintainability  research.  For  example,  one 
can  expect  a  correlation  between  the  accessibility  of  components  and  the 
complexity  of  maintenance  procedures.  Without  knowing  the  relative  merits  of 
various  design  variables,  it  would  be  difficult  to  construct  a  model  which 
yields  data  for  design  decisions.  It  has  been  shown  that  correlational 
methods  have  certain  weaknesses  and  it  v/ill  take  some  work  to  improve  them 
before  they  v/ill  be  ideally  suited  to  our  needs.  In  the  meantime,  one  of  the 
major  emphases  will  be  to  define  the  design  variables  quantitatively  in 
psychometric  terms  and  in  terms  useful  to  the  engineer. 

Laboratory  Experiments 

While  time-synthesis  methods  and  correlational  methods  have  shown  some 
success  in  predicting  maintainability  of  a  design  choice,  these  methods  do  not 
provide  design  engineers  v/i th  information  as  to  how  to  reduce  maintenance 
workload  by  way  of  equipment  design  and  further,  what  an  alternative  design 
might  be.  This  deficiency  results  from  a  lack  of  knowledge  of  the  processes 
or  mechanisms  through  which  design  factors  make  an  influence.  Recently, 
investigators  began  to  address  this  issue  through  controlled  experiments  in 
laboratory  settings. 

In  a  laboratory  experiment,  a  maintenance  task  is  broken  down  into  several 
behavioral  components  and  processes,  i.e.,  psychomotor,  conceptual,  and 
perceptual.  An  experiment  is  then  designed  to  investigate  the  impact  of 
design  factors  on  one  piece  of  the  behavioral  components  or  processes.  Quite 
often,  the  removal  and  installation  processes  are  examined  in  a  psychomotor 
skill  domain.  The  troubleshooting  and  checkout  processes  are  treated  as 
cognitive  processes;  some  times  as  a  problem  solving  process,  other  times  as  a 
perceptual  process. 

Recent  studies  have  concentrated  on  investigation  of  troubleshooting 
processes  and  the  development  of  optimal  troubleshooting  strategies  because 
troubleshooting  consumes  the  majority  of  maintenance  time.  A  family  of 
troubleshooting  strategies  has  been  devised.  The  effectiveness  of  these 
strategies  will  be  examined  across  different  equipment  designs. 


The  major  deficiency  of  laboratory  experiments  is  the  general izability  of 
laboratory  findings  to  the  real  world  and  the  lack  of  their  acceptance  by 
field  maintainers.  As  Christensen  and  Howard  (1981)  pointed  out,  none  of  the 
laboratory-derived  troubleshooting  strategies  has  been  adopted  by 
maintainers.  One  problem  of  laboratory  experiments  is  that  the  environmental 
fidelity  is  often  ignored.  In  a  laboratory  setting,  environmental  variables 
are  often  controlled.  However,  one  can  expect  that  there  is  an  interaction, 
which  is  very  important,  between  design  variables  and  environmental 
variables.  From  our  interviews  with  NAMTRADET  instructors,  it  was  found  that 
red  color-coding  is  ineffective  for  organizational  level  maintenance 
performance  because  red  lighting  is  used  on  the  hanger  deck  of  a  carrier. 
Another  finding  is  that  limited  workspace  on  a  carrier  tends  to  change  the 
maintenance  task  structure.  If  investigators  ignore  these  interactions  in  the 
working  environment,  the  laboratory  findings  may  be  nullified  or  even  reversed. 

The  second  problem  with  laboratory  experiments  is  related  to  the 
measurement  and  definition  of  design  variables.  Rigney  (1977)  complained  that 
"many  of  the  design  variables  have  not  been  suitably  identified  and  many 
others  have  not  been  measured  in  an  appropriate  way"  (Goldman  &  Slattery, 

1977,  pp.  254).  With  regard  to  the  measurement  of  design  variables,  if  the 
accessibility  of  the  components  is  defined  as  the  number  of  steps  needed  to 
reach  the  components,  then  the  definition  assumes  that  the  distance  between 
every  two  steps  is  equal,  as  in  an  equal  interval  scale.  However,  we  know 
that  this  assumption  may  not  hold  true  in  the  real  world.  Let  us  take  the 
removal  of  an  F-14  Sensing  Element  #1  as  an  example.  In  removing  Sensing 
Element  #1,  the  overwing  fairing  needs  to  be  removed  so  that  the  sensing 
element  can  be  accessed.  If  we  define  the  accessibility  of  the  sensing 
element  as  the  number  of  steps  taken  in  removing  the  overwing  fairing,  then  we 
assume  that  the  step  of  loosening  two  aft  screws,  and  one  step  in  removing  the 
overwing  fairing,  is  equal  to  the  step  of  removing  eight  screws.  These  two 
steps,  of  course,  cannot  be  regarded  as  equal.  On  the  other  hand,  if  the 
number  of  steps  is  measured  on  an  ordinal  scale,  a  two-step  action  is  not 
always  less  tnan  another  three-step  action.  Another  point  is  that  some  design 
variables  may  need  to  be  defined  from  the  viewpoint  of  the  human-machine 
interface.  On  the  issue  of  the  complexity  of  design.  Rouse  and  Rouse  (1979) 
proposed  that  the  definition  of  complexity,  within  the  context  of 
troubleshooting  tasks,  should  deal  with  how  much  maintainers  understand  the 
concepts  of  problem  and  solutions  strategy,  as  well  as  properties  of  the 
problem  Itself.  They  tested  the  validity  of  four  measures  of  complexity:  (1) 
one  based  on  the  number  of  components  In  the  system,  (2)  one  based  on 
computational  complexity,  (3)  one  based  on  the  number  of  relevant 
relationships,  (4)  one  based  on  Information  theory.  It  was  found  that  the 
last  two  measures  are  good  predictors  of  troubleshooting  performance  (i.e., 
troubleshooting  time).  Therefore,  they  concluded  that  psychological 
perspectives  should  be  Incorporated  into  the  definition  and  measurement  of 
complexity.  In  this  regard,  a  factor  analysis  and  multidimensional  scaling 
may  be  a  viable  method  for  identifying  and  developing  design  variable  measures. 

Furthermore,  a  third  problem  related  to  laboratory  research  is  performance 
measures.  In  developing  performance  measures,  several  should  be  taken  into 
account.  First,  performance  measures  employed  in  a  laboratory  experiment  must 
be  related  to  system  criteria.  At  least,  the  relationship  between  maintenance 
performance  measures  used  in  a  laboratory  setting  and  system  effectiveness 


criteria  should  be  established.  Thus,  the  laboratory  data  can  be  transformed 
into  the  system  engineering  domain  and  accepted  by  engineers.  Secondly,  the 
interrelationships  among  performance  measures  should  be  examined.  If  two 
performance  measures  are  independent  of  each  other,  this  Implies  that  we  are 
dealing  with  two  different  processes  rather  than  one.  In  the  Potempa  et  al. 
study  (1975),  different  sets  of  design  features  were  found  to  be  related  to 
maintenance  time,  and  errors,  respectively.  This  finding  may  indicate  that 
the  process  of  producing  maintenance  time  is  different  from  that  of 
maintenance  errors.  Therefore,  one  should  be  cautious  in  generalizing  the 
data  from  one  study  to  another.  One  way  to  avoid  this  problem  is  to  employ  a 
multivariate  analysis  in  dealing  with  several  discernible  aspects  of 
maintenance  performance  at  the  same  time.  In  this  way,  maintenance 
performance  can  be  Investigated  as  a  whole  contruct  rather  than  a  construct 
torn  into  pieces.  Finally,  the  measurement  of  maintenance  performance  must  be 
developed  in  a  way  that  will  reflect  the  impact  of  design  on  maintainers' 
capabilities  and  their  limitations.  A  task  analysis  may  assist  in  the 
development  of  performance  measures,  especially  in  defining  and  classifying 
maintenance  errors.  The  requirements  and  procedures  of  maintenance  task 
analyses,  however,  need  to  be  specified. 

From  the  discussion  of  problems  with  laboratory  experiments,  some 
Improvements  need  to  be  made  in  relating  laboratory  data  to  field  data.  The 
first  step  of  this  job  will  be  to  observe  and  analyze  how  maintainers  perform 
in  a  field  setting  to  aid  in  identifying  realistic  experimental  variables  and 
in  developing  a  maintenance  performance  measurement  scheme.  Unless  the 
important  potential  independent  variables  are  suitably  identified,  it  is  not 
possible  to  investigate  design  effects  on  maintenance  performance  in  a 
well -control led  laboratory  setting. 

Discussion  and  Conclusion 

In  the  preceding  discussion  of  current  approaches  to  maintainability 
research,  one  can  see  that  there  is  a  need  to  develop  a  procedure  which  will 
enable  us  to  identify  current  maintenance  problems  and  improve  maintainability 
of  future  equipment.  In  the  Identification  of  current  maintenance  problems 
and  the  derivation  of  solutions  to  these  problems,  a  procedure  is  proposed. 
This  procedure  (1)  includes  3-M  data  analysis,  (2)  discusses  high 
maintenance-man-hour  systems  with  maintenance  personnel,  (3)  conducts  a 
maintenance  task  analysis,  (4)  performs  field  observations,  and  (5)  documents 
problems  and  develops  recommendations  for  change.  Review  of  3-M  data  yields 
some  indication  as  to  which  maintenance  tasks  should  be  looked  at.  Having 
determined  candidate  maintenance  tasks,  structured  interviews  with  maintenance 
personnel  can  shed  some  light  on  the  characteristics  and  locus  of  maintenance 
problems  in  a  particular  maintenance  task  and  provide  an  important  data  source 
of  design  deficiencies.  A  maintenance  task  analysis  further  gives  information 
concerning  characteristics  of  basic  parameters  of  maintenance  performance  and 
critical  design  variables.  Moreover,  field  observations  yield  data  to  compare 
field  maintenance  performance  with  the  maintenance  task  structure.  These 
observations  together  with  structured  interviews,  in  turn,  provide  some 
Insight  into  design  rules  for  the  particular  equipment  and/or  component. 

For  the  improvement  of  the  maintainability  of  future  equipment,  design 
tools  (i.e.,  design  guidelines  and  a  simulation  model)  which  can  be  used  in 


the  early  design  phase,  need  to  be  developed.  The  first  step  in  developing 
these  design  tools  will  be  the  establishment  of  a  maintenance  performance  data 
base.  The  data  base  will  comprise  two  types  of  empirical  data.  One  type  will 
contain  data  on  the  extent  to  which  critical  design  variables  affect 
maintenance  performance  in  a  particular  system,  which  in  turn  affect  system 
effectiveness.  The  other  will  contain  information  on  how  a  design  factor 
Interacts  with  other  factors  and  how  we  may  characterize  this  interactive 
effect  so  that  a  matrix  of  favorable  design  factors  and  design  guidelines  can 
then  be  derived.  Therefore,  the  construction  of  such  a  data  base  will  require 
a  series  of  parametric  experimentations.  Before  the  construction  of  this  data 
base,  however,  more  precise  and  sensitive  measurements  of  maintenance 
performance,  as  well  as  design  variables,  must  be  developed.  In  this  vein,  a 
task  analysis  can  contribute  greatly  to  the  development  of  maintenance 
performance  measurement.  First,  a  task  analysis  can  be  used  as  a  vehicle  to 
Identify  specific  behavioral  components  which  should  be  examined.  Secondly,  a 
task  analysis  would  shed  some  light  on  characteristics  of  basic  parameters  of 
maintenance  performance  and  their  relationships  to  system  effectiveness. 
Therefore,  it  can  provide  a  basis  for  relating  field  data  to  laboratory  data. 
With  respect  to  the  measurement  of  design  variables,  psychological  dimensions 
underlying  physical  characteristics  should  be  identified.  In  this  light, 
factor-analytic  methods  and  multidimensional  scaling  procedures  can  help  in 
determining  a  set  of  psychological  factors  that  underlie  physical  layouts.  In 
order  to  assure  the  general Izability  of  the  data  base,  a  functional  taxonomy 
of  maintenance  tasks  In  various  types  of  equipment  should  be  examined.  The 
functional  taxonomy  will  document  comnonalities  and  dissimilarities  in 
essential  skills  and  knowledge  required  to  maintain  different  equipment.  This 
information  will  then  enable  design  engineers  to  decide  the  extent  to  which 
the  data  may  be  applied  to  their  system,  viz.,  its  general izability. 

Finally,  it  must  be  remembered  that  when  we  have  developed 
deslgn-for-maintainers  methodologies,  design  engineers  are  likely  not  to  adopt 
our  recommendations  if  Information  documented  in  the  maintenance  performance 
data  base  Is  too  massive  to  be  handled  by  design  engineers  or  if  the  process 
of  applying  maintainability  data  and  design  principles  is  too  complicated. 
Thus,  we  must  also  consider  design-for-deslgners.  We  must  develop  ways  to 
reduce  designers'  workload.  That  is,  we  need  to  develop  a  decision  aid  for 
design  engineers  to  Incorporate  maintainability  data  and  design  guidelines 
into  their  design.  The  decision  aid  could  be  a  computer  "expert  system" 
derived  from  artificial  intelligence  principles.  The  expert  system,  together 
with  a  maintenance  performance  simulation  model,  could  act  as  an  intelligent 
assistant,  providing  advice  and  exercising  trade-offs  in  the  design  process. 

By  interacting  with  the  machine  "expert,"  design  engineers  could  achieve  a 
design  solution  which  would  optimize  both  the  reliability  and  maintainability 
of  a  weapon  system. 
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Purpose 

This  paper  assesses  the  possibility  of  using  data  generated  by  the 
maintenance  management  systems  to  evaluate  the  effectiveness  of  alternative 
methods  of  training  military  maintenance  personnel. 

Background 

Costs  of  training  maintenance  skills  comprise  a  significant  portion  of  the 
$3  billion  spent  each  year  for  technical  training  at  military  schools  and  can 
be  expected  to  increase  with  increases  in  the  complexity  of  weapon  systems. 

On  the  other  hand,  the  potential  costs  of  "inadequate"  maintenance,  in  terms 
of  increased  operating  costs  and  reduced  operational  capabilities,  may  be 
considerably  greater  than  the  costs  of  providing  more  extensive  and  more 
effective  maintenance  training. 

The  effectiveness  of  training  is  measured  currently  mainly  by  student 
achievement  at  school.  Occasional  surveys,  where  supervisors  rate  the  job 
performance  of  recent  trainees,  can  only  provide  subjective,  rather  than 
objective  data;  moreover,  such  surveys  generally  provide  data  only  on  limited 
rather  than  on  systematic  samples  of  trainees.  However,  the  true 
effectiveness  of  training  lies  in  the  performance  of  personnel  on  the  job, 
rather  than  at  school,  and  the  comparative  effectiveness  of  different  amounts 
and  methods  of  training  should  be  measured  by  comparing  on-the-job 
performances  of  personnel  trained  in  different  ways. 

Correlations  between  school  achievement  and  on-the-job  performance  of 
maintenance  personnel  have  not  been  established,  and  the  development  and 
operation  of  a  data  system  for  this  purpose  would  be  a  costly  undertaking. 
However,  the  military  services  currently  employ  extensive  systems  for  the 
day-to-day  management  of  their  maintenance  operations,  and  these  systems 
generate  extensive  historical  data.  If  these  data  could  be  used  to  shed  light 
on  the  performance  of  either  maintenance  organizations  or  the  individuals 
assigned  to  them,  they  might  also  provide  information  that  would  be  helpful  in 
determining  the  effectiveness  of  alternative  methods  of  maintenance  training. 
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Sources  of  Maintenance  Performance  Data 


The  military  services  operate  five  maintenance  management  systems;  these 
are  identified,  together  with  their  short  titles,  in  Table  1.  Taken  together, 
these  systems  encompass  organizational  and  intermediate  maintenance  of  all 
military  aircraft,  all  Army  and  Air  Force  ground  equipment  (including 
missiles),  and  all  Navy  ships  and  shipboard  equipment  (except  nuclear 
missiles).  The  Air  Force  66-1  and  66-5  systems  employ  the  same  reporting 
formats  and  codes;  only  the  Tactical  Air  Forces  use  the  66-5  system  while  all 
other  Air  Force  organizations  use  the  66-1  system. 

We  should  not  be  surprised  to  find  that  none  of  these  systems,  at  least  in 
its  present  form,  provides  a  suitable  vehicle  for  assessing  the  effectiveness 
of  training.  The  reasons  for  this  conclusion  lie  in  two  different,  but 
related,  considerations.  The  first  encompasses  rather  severe  restrictions  on 
the  way  maintenance  must  be  documented  in  order  to  translate  the  data  into 
measures  of  training  effectiveness.  The  second  concerns  ways  in  which  certain 
characteristics  of  current  military  operations,  maintenance  practices,  and 
equipment  may  interfere  with  attempts  to  assess  training  effectiveness.  It 
should  be  noted,  however,  that  these  systems  were  designed  to  manage 
maintenance  operations  and  were  not  meant  to  be  used  to  evaluate  the 
effectiveness  of  training  or  other  aspects  of  human  performance. 

Table  1 

Maintenance  Management  Data  Systems  Used  by  the  Military  Services 


Service 


Maintenance  Management  System _ 

Name  Short  Title 


Army  The  Army  Maintenance  Management  System 

Navy  Naval  Ships'  Maintenance  and  Material 

Management  System 

Navy  Naval  Aviation  Maintenance  and 

Material  Management  System 


Air  Force  Air  Force  Maintenance  Management 
Systems 


TAMMS 
Ships'  3-M 


Aviation  3-M 


66-1  and  66-5 


Characteristics  of  Data  Needed  to  Assess  Training 

The  effectiveness  of  alternative  methods  of  training  maintenance  personnel 
can  be  evaluated  by  comparing  how  well  personnel  trained  two  different  ways 
maintain  the  same  types  of  equipment  in  the  field.  However,  we  have  to  be 
reasonably  sure  that  the  personnel  in  both  groups  were  actually  performing  the 
same  types  of  maintenance  on  the  same  equipments.  Verifying  that  these 
conditions  are  met  places  a  series  of  constraints  on  the  data  developed 
through  the  management  system,  as  follows: 


•  The  data  must  measure  the  outcome  of  maintenance  operations  in  terms 
that  provide  a  criterion  of  maintenance  personnel  performance. 


•  The  data  must  provide  unambiguous  (i.e.,  coded)  answers  to  four 
questions  regarding  each  maintenance  operation: 

--  What  equipment  was  maintained? 

—  Why  was  maintenance  required  (i.e.,  the  nature  of  the 
equipment  malfunction)? 

--  What  was  done  to  it  (i.e.,  the  nature  of  maintenance 
performed)? 

--  Who  performed  the  maintenance? 

•  The  data  must  separately  document  small-scale  discrete  and 
well-defined  maintenance  tasks  (i.e.,  tasks  that  are  comparable 
whenever  they  are  performed  on  the  same  subsystem  or  assembly  or 
black  box  installed  on  the  same  model  of  equipment  end-item,  such  as, 
remove,  adjust)  rather  than  completed  maintenance  that  results  in 
equipment  being  returned  to  operational  status. 

•  The  data  must  identify  the  equipment  maintained  at  a  sufficiently  low 
level  (e.g.,  subsystem  or  assembly)  so  that  it  can  be  associated  with 
a  single  skill  area  related  to  a  specific  training  program. 

•  The  data  must  encompass  a  sufficiently  wide  set  of  maintenance  tasks 
to  provide  a  representative  sample  of  the  on-the-job  skill 
requirements  of  a  particular  skill  area. 

•  The  data  must  identify  individuals  (or  organizations)  performing 
maintenance  in  a  way  that  will  allow  the  association  of  particular 
skill  areas  on  the  job  with  specific  training  programs. 

There  are  four  characteristics  of  military  equipment,  maintenance 
organizations,  and  maintenance  practices  that,  where  they  occur,  are 
impediments  to  assessing  the  effectiveness  of  training  by  using  on-the-job 
performance  data,  as  follows: 

t  Maintenance  tasks  may  be  performed  by  a  group  (or  team)  of  personnel 
rather  than  only  by  individuals  (i.e.,  team  maintenance). 

•  Maintenance  tasks  associated  with  one  skill  area  may  be  performed  by 
personnel  trained  in  a  different  skill  area  (i.e.,  cross-skill 

mai ntenance) . 

•  Maintenance  organizations  may  not  be  organized  internally  into 
ski 11 -related  Work  Centers. 

•  Hot  all  military  end-items  or  their  installed  subsystems  are  built  to 
standard  configurations. 

These  features  are  intractable  constraints  on  assessing  the  effectiveness  of 
training  from  on-the-job  performance  data. 

However,  maintenance  reporting  systems  might  be  designed  (or  current 
systems  modified)  to  identify  where  (i.e.,  on  which  maintenance  tasks) 
specific  activities  occur.  Then,  maintenance  activity  not  associated  with 


these  constraints  night  shed  light  on  training  effectiveness.  A  sunnary  of 
the  characteristics  of  the  data  provided  by  the  maintenance  management 
systems,  with  regard  to  the  issues  discussed  above,  appears  in  Table  2. 

Assessment  of  Maintenance  Management  Systems 

Table  3  denotes,  for  each  maintenance  management  system,  the  extent  to 
which  current  documentation  provides  data  suitable  for  assessing  the 
effectiveness  of  training  or  where  structural  inconsistencies  in  the  data  may 
prevent  such  assessments.  It  was  developed  through  study  of  the  documentation 
provided  for  these  systems  supplemented  by  discussions  with  knowledgeable 
service  personnel.  Note  that  a  notation  in  the  upper  part  of  the  table  shows 
an  impediment  to  assessing  training  effectiveness,  while  a  notation  in  the 
lower  part  signifies  data  that  permit  (or  support)  assessments.  For  both  sets 
of  characteristics,  the  notations  in  the  table  give  the  maintenance  systems 
(and  the  documentation  they  generate)  the  benefit  of  the  doubt  regarding  their 
capabilities  for  assessing  training  effectiveness.  That  is,  where 
uncertainties  remained,  it  was  assumed  that  the  requisite  conditions  for 
assessing  training  effectiveness  v/ere  met. 

TAMMS 


TAMMS  appears  to  provide  no  capability  for  assessing  the  effectiveness  of 
training.  The  Army  practices  both  team  and  cross-skill  maintenance  in 
peacetime  because  Army  units  will  operate  in  that  fashion  under  combat 
conditions.  A  major  portion  of  Army  maintenance  units  are  not  structured 
further  into  Work  Centers  (WC)  so  that  there  is  no  way  to  identify  the  skill 
areas  of  personnel  performing  maintenance.  In  addition,  the  maintenance 
reporting  format  has  no  provision  for  noting  where  team  maintenance  occurs. 

Ships'  3-M 


The  Ship's  3-M  system  appears  to  provide  no  capability  for  assessing 
maintenance  performance  (and,  hence,  training  effectiveness)  for  reasons  that 
encompass  both  the  nonstandard  nature  of  shipboard  equipment  and  the  data 
reported.  Except  in  the  area  of  electronics  and  ordnance,  shipboard  systems 
are  not  standardized,  and  ships  are  not  outfitted  to  standard  configurations. 
Even  if  the  configuration  problem  was  not  present,  the  data  reported  through 
the  Ships'  3-M  system  appear  inconsistent  with  assessing  training 
effectiveness  of  three  counts.  First,  maintenance  reporting  is  notably 
incomplete  and  cannot  be  assumed  to  provide  representative  samples  of  the 
ranges  of  skills  for  which  personnel  are  trained.  Second,  data  are  reported 
only  for  complete  maintenance  actions  (as  opposed  to  individual  maintenance 
tasks).  Finally,  the  English  descriptions  and  code  systems  used  to  document 
maintenance  are  inadequate  to  identify  comparable  maintenance  actions. 

Aviation  3-M,  66-1,  and  66-5 


Even  though  the  Air  Force  systems  (66-1  and  66-5)  employ  the  same 
data-reporting  forms,  they  display  quite  different  potentials  for  assessing 
the  effectiveness  of  training.  The  66-5  system,  that  is  employed  only  in 
conjunction  with  the  Production  Oriented  Maintenance  Organization  (POMO) 
concept,  appears  to  provide  no  potential  for  this  assessment.  Under  the  POMO 
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Table  2 


Characteristics  of  Data  Created  by  Maintenance  Management  Systems 


Maintenance  Management  System 

Characteristics  of  Data 

TAMMS 

(Army) 

Ships'  3-M 
(Navy) 

Aviation  3-M 
(Navy) 

66-1/66-5 
(Air  Force) 

Applicable  equipment 

All 

All 

All 

All 

equipment 

ships 

aircraft4 

equipment 

Extent  of  maintenance  activity  documentation 

Total 

Selected  Types 
of  Maintenance 

Total 

Total 

Central  reporting  of  recorded  data 

None 

Total 

Total 

Total 

Level  of  documentation 

Maintenance 

Maintenance 

Maintenance 

Maintenance 

Data  describes  (or  answers) 

Task 

Action0 

Task 

Task 

What  equipment,  at: 

Major  system  level 

- 

C 

- 

- 

Subsystem  or  component  level 

C 

- 

C 

C 

Why  maintenance  was  required 

C 

E 

C 

C 

What  was  done  to  It 

c 

E 

c 

c 

Who  did  It: 

Individual 

- 

- 

Ed 

- 

Work  center  (or  ski  11 -related  shop) 

- 

C 

c 

Maintenance  organization  (or  more 
than  one  work  center) 

c 

lc' 

” 

Note:  C  =  Coded;  E  =  English 

includes  aircraft,  air-launched  missiles,  support  equipment,  training  equipment. 

includes  aircraft;  ground  and  air-launched  missiles;  precision  measuring  and  other  support  equipment; 
training  equipment;  ground  communications,  electronics,  and  meteorological  equipment. 

cGroup  of  maintenance  tasks. 

^Individuals  performing  maintenance  are  identified  by  name  only  In  Initial  hardcopy  forms  that  are  retained 
by  local  units  for  a  limited  time. 

eAl 1  work  centers  Involved  In  a  maintenance  action  are  Identified  In  one  record. 
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Table  3 

Assessment  of  Maintenance  Management  Systems 


TAMMS 

(Army) 


Maintenance 
Management  System 
Ships  I  Aviation 
3-M  3-M 

(Navv)  (Navv) 


66-1/66-5 

fUSAF) 


Structural  inconsistencies  present 


Team  Maintenance 

Cross-skill  Maintenance 

Maintenance  Organizations  Not 
Structured  According  to 
Ski 11 -Area  (i.e.  WCs) 

Non-standardized  Equipment 


X 

X 

X 

a 

a 

Data  requirements  satisfied 


Quantifies  a  Criterion  of 
Performance 

Coded  Descriptions  of  Maintenance 
Performed 

•  What  Equipment  was  Maintained? 

•  What  was  wrong  with  it? 

•  What  was  done  to  it? 

•  Who  Performed  the  Maintenance? 

Maintenance  ‘Task"  Documentation 

Equipment  can  be  Identified  with 
Skill  Area 

Recorded  Maintenance  Representa¬ 
tive  of  Job  Requirements^ 

Who  Performed  Maintenance  can  be 
Identified  with  Skilsl-Area 


Work  Center  Code 

Some  WCs  are  manned  by  personnel  with  training  in  several  skill  areas. 

Recording  refers  only  to  initial  documentation  of  maintenance  performed. 

A  comprehensive  capability  for  assessing  training  effectiveness  would 
require  that  these  data  be  reported  to  higher  echelons. 

More  than  one  WC,  in  the  same  skill  area,  may  be  involved  in  and  iden¬ 
tified  with  the  documentation  of  one  maintenance  action. 

Except  for  WCs  manned  by  personnel  with  training  in  different  skill  areas. 


structure,  followed  by  the  Tactical  Air  Forces,  the  large  majority  of 
maintenance  personnel  are  assigned  to  the  organizational  echelon  where  the 
development  of  cross-skill  capabilities  is  a  primary  POMO  policy.  This  policy 
might  be  implemented  in  several  ways  (e.g.,  by  forming  inter-skill  maintenance 
teams,  by  assigning  personnel  to  Work  Centers  other  than  those  for  which  they 
have  been  trained).  Maintenance  data  reporting  in  the  66-5  system  appears  to 
provide  no  way  to  identify  cross-skill  work  that  is  promoted  by  these  means. 

In  addition,  it  is  possible  that  cross-skill  maintenance  might  be  prevalent  to 
the  point  where  a  satisfactory  sample  of  primary-skill  maintenance  could  not 
be  obtained. 

The  66-1  and  Aviation  3-M  systems  may  provide  a  restricted  capability  for 
assessing  training  effectiveness  if  occurrences  of  team  and  cross-skill 
maintenance  could  be  identified  unambiguously  and  thus  eliminated  from 
analyses.  This  might  be  possible  by  making  some  seemingly  modest  changes  to 
the  data  reported  and  by  supplementing  these  data  with  information  that  is 
normally  available  in  unit  roster  and  personnel  record  systems.  Whether  the 
reporting  system  changes  and  supplementary  data  suggested  here  will,  in  fact, 
provide  this  identification  requires  inquiry  into  maintenance  operations  to  a 
depth  not  accomplished  in  this  study. 

Team  maintenance.  The  66-1  system  currently  documents  maintenance  "crew 
si ze"  and  "start/stop"  times  and  notes  all  crew  changes  and  work  interruptions 
that  occur  during  the  course  of  a  maintenance  task.  These  data  appear  to 
provide  a  satisfactory  separation  of  team  from  individually  performed  tasks. 
However,  it  is  not  clear  whether  the  separation  provided  by  these  data  (along 
with  other  data  contained  in  maintenance  records)  is  reliable  for  all  possible 
types  of  maintenance  and  the  various  conditions  under  which  they  may  be 
performed  (e.g.,  shift  changes,  interruptions  for  lack  of  parts,  changes  in 
malfunction  diagnosis).  Aviation  3-M  currently  provides  no  permanently 
retained  information  regarding  the  number  of  individuals  contributing  to  a 
maintenance  task.  However,  the  similarities  in  equipments,  their  maintenance 
requirements,  and  maintenance  organization  between  USAF  and  Naval  aircraft 
appear  sufficient  to  suggest  that  whatever  data  and  formats  would  identify 
team  maintenance  in  the  66-1  system  would  serve  the  same  purpose  in  the 
Aviation  3-M  system. 

Cross-skill  maintenance.  Both  equipments  and  Work  Centers  (WC)  appear 
relatable  to  skill  areas  in  the  Aviation  3-M  and  66-1  systems  so  that 
occurrences  of  cross-skill  work  should  be  identifiable.  However,  a  number  of 
exceptions  were  noted  in  examples  contained  in  the  Aviation  3-M  user  manuals 
that  cast  doubt  on  the  validity  of  this  conclusion.  The  extent  to  which  WCs, 
in  fact,  specialize  in  one  skill  area:  (1)  may  vary  among  maintenance  or 
organizations  as  functions  of  size,  equipment  maintained,  and  command 
decision,  (2)  will  vary  among  the  different  WCs  within  maintenance 
organizations,  and  (3)  may  vary  over  time  within  the  same  WC  as  a  function  of 
service-wide  personnel  availabilities.  Further,  as  a  result  of  workload 
variations,  personnel  may  be  transferred  temporarily  between  WCs  that  are 
associated  with  quite  different  skill  areas.  Similarly,  the  extent  to  which 
skill  areas  can  be  associated  with  WCs  also  varies.  All  such  variations, 
whether  thay  constitute  normal  practices  or  exceptions,  weaken  the  case  for 
Identifying  cross-skill  maintenance,  especially  where  the  variations  are 
systematic. 


Reporting  system  changes.  The  uncertain  identification  of  both  team  and 
cross-skill  maintenance  actions  could  be  reduced  if  data  records  were  to 
identify  specifically  all  personnel  who  performed  in  a  maintenance  task. 

While  identifying  personnel  in  the  central  maintenance  data  files  conflicts 
with  provisions  of  the  Privacy  Act,  it  should  be  possible  to  name  individuals 
in  local  unit  records  and  then  to  process  these  records  in  ways  that  would 
provide  suitable  identification  for  analyses  of  training  effectiveness  while 
being  consistent  with  the  Act.  For  example,  individuals  could  be  preselected 
for  analysis  of  performance  on  the  basis  of  training  and  experience 
Information  in  personnel  records.  Look-up  tables  could  be  established  or 
special  notations  (flags)  could  be  attached  to  their  names  or  identification 
numbers  as  a  device  for  identifying  the  maintenance  tasks  they  subsequently 
perform.  Where  maintenance  documentation  identified  these  individuals  and 
where  neither  team  nor  cross-skill  maintenance  was  indicated,  the  records 
would  be  duplicated  for  analysis  of  performance,  and  the  identification  of 
individuals  could  then  be  deleted  from  the  maintenance  records  submitted  to 
central  files. 

Analyses  of  training  effectiveness  could  then  be  performed  without 
interfering  with  either  the  current  maintenance  management  systems  or  the 
organization  of  maintenance.  Automatic  data  processing  of  the  Aviation  3-M 
system  is  currently  being  modified  to  accomplish  on-line  record  entry  and 
updating  (instead  of  hardcopy  and  punched  card)  as  part  of  the  Navy  Air 
Logistics  Command  Management  Information  System  (NALCCMIS)  program.  As 
presently  designed,  local  unit  on-line  records  will  identify  personnel  by 
name,  and  this  information  will  be  dropped  when  the  local  records  are 
committed  to  the  Aviation  3-M  central  tape  files.  It  appears  to  be  a  feasible 
further  step  to  provide  identification  of  personnel  in  a  way  that  would  allow 
selection  of  the  on-line  records  for  analyses  of  their  performance. 

Even  with  these  changes,  a  question  would  remain  as  to  whether  the 
resulting  samples  of  maintenance  performance  would  be  representative  of  the 
job  requirements  of  skill  areas.  The  prospects  are  not  promising,  especially 
at  the  organizational  echelon.  An  analysis  that  concentrated  on  recently 
trained  personnel  would  be  limited  essentially  to  personnel  who  are  not  yet 
qualified  for  independent  work  on  aircraft  equipment.  The  Air  Force  has  a 
service-wide  procedure  for  job  qualification  that  consists  of  structured  OJT 
and  performance  examinations,  and  personnel  are  not  permitted  to  perform 
independent  work  until  specifically  qualified.  The  Navy  also  has  defined 
qualifications  standards  and  OJT  programs,  although  the  Navy  program  is  less 
restrictive  regarding  the  work  which  can  be  accomplished  by  new  personnel.  It 
could  well  be  that  by  the  time  an  individual  meets  the  qualification  standards 
for  independent  work  the  impact  of  different  formal  training  programs  would  be 
diluted  to  the  point  where  initial  differences  in  performance  that  were  due  to 
the  training  would  be  washed  out. 
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Maintenance  activity  is  a  function  of  three  primary  factors:  the  human 
performer,  the  environment  in  which  the  activity  is  performed,  and  the  system 
being  restored  or  adjusted. 

The  maintainer' s  capabilities  are  determined  by  many  factors  including  (1) 
his  innate  abilities,  (2)  his  training,  (3)  the  type,  recency,  and  amount  of 
experience,  and  (4)  his  motivation.  The  ease  of  performing  the  task  can  be 
greatly  affected  by  the  environment  in  which  it  is  attempted.  These  factors 
include  (1)  time  constraints,  (2)  availability  and  quality  of  test  equipment, 
and  (3)  ambient  conditions,  such  as  space,  temperature,  stability  and 
visibility. 

The  characteristics  of  the  system  itself,  however,  dictate  the  inherent 
difficulty  of  the  maintenance  task.  The  design  of  the  man-machine  interface, 
which  may  include  switches,  dials,  controls,  and  test  points,  partially 
determines  the  ease  with  which  information  about  the  system  can  be  obtained. 
The  physical  packaging  affects  the  ease  of  accessing  internal  elements  for 
diagnostic,  repair,  and  replacement  purposes. 

The  ultimate  design  tool  would  constructively  guide  the  designer  as 
decisions  and  trade-offs  are  considered.  Unfortunately,  the  relationships 
between  maintainability  and  system  design  are  not  yet  well  defined  and 
quantified.  Consequently,  even  automated  techniques  cannot  currently  take  on 
creative  design  functions  with  maintainability  as  the  design  criterion.  An 
achievable  alternative,  however,  would  be  a  sensitive  assessment  process  which 
can  evaluate  a  design  specification  in  terms  of  its  projected  maintainability 
in  some  defined  environment.  Such  a  process  could  be  used  by  the  designer  to 
determine  the  impact  of  alternatives  under  consideration,  and  it  could  be  used 
to  compare  competing  specifications  for  complex  systems.  Continued  use  of 
such  an  assessment  technique  could  ultimately  yield  design  principles  and 
relationships  to  form  the  foundation  of  a  constructive  design  aid. 

This  paper  will  first  summarize  existing  approaches  to  maintainability 
assessment,  and  will  then  describe  an  analytical  approach  currently  under 
development  by  this  laboratory  (Towne,  Fehling,  4  Bond,  1981). 


Techniques  for  Analyzing  Maintenance  Workload 

The  techniques  which  have  emerged  to  date  are  only  moderately  successful 
in  producing  repair  time  estimates  which  correlate  with  actual  repair  time 
data.  Unfortunately,  the  existing  techniques  tend  to  be  specific  to 
particular  technologies  or  maintenance  settings,  they  tend  to  offer  little 
insight  to  the  designer,  and  most  tell  nothing  about  the  performance  required 
of  the  maintainer.  These  maintainability  prediction  methods  can  be  classified 
into  the  following  six  categories. 

Empirical  Extrapolation 

For  a  new  radar  system,  one  might  predict  that  maintainability 
requirements  will  be  about  like  they  were  on  an  old  radar  system  that  is 
similar  to  the  new  one.  Of  course,  it  may  be  hard  to  say  just  how  similar  the 
new  item  is  to  the  previous  model,  but  a  rough  similarity  rule  may  still  be 
practically  useful.  At  least  the  real-experience  data  should  introduce  some 
realism  into  expectations  for  the  new  system. 

One  possible  empirical  generalization  is  that  variance  in  repair  times 
among  military  equipments  is  largely  due  to  the  maintenance  concept  employed. 
Airborne  radars  and  radios  are  serviced  via  module  replacement  policy,  whereas 
ship  and  ground-based  items  may  require  troubleshooting  and  repair  down  to  the 
piece-part  level.  Hence,  standard  deviations  for  airborne  equipment  are  on 
the  order  of  half  an  hour,  as  compared  to  about  one  and  a  half  hours  for  large 
ground  and  ship  systems. 


Checklist  Methods 


Many  factors  are  known  to  facilitate  preventive  and  corrective  maintenance 
tasks.  Clearly,  if  some  key  test  points  are  inaccessible,  unlabeled,  or 
otherwise  difficult  to  use,  then  the  equipment  will  be  harder  to  service. 

Lists  of  good  design  and  support  features  have  been  assembled,  with  the  idea 
of  scoring  a  system  on  the  various  criteria.  The  famous  Munger-Wi 1 1  is  list 
gave  241  design  features  which  had  potential  significance  for  maintainability 
(Munger  &  Willis,  1959).  A  more  manageable  scheme  derives  from  MIL-HDBK-472 
(U.S.  Department  of  Defense,  1966).  There  are  three  design  checklists  in  the 
document  concerned  with:  (1)  physical  features  such  as  access  to  and  display 
of  information,  the  types  of  fault  indicators,  safety  considerations,  and  so 
forth,  (2)  the  need  for  external  facilities  (special  equipment,  etc.),  and  (3) 
the  personnel  requirements  for  successful  maintenance.  According  to  some 
trials  at  RCA-Camden,  reasonable,  slightly  optimistic,  predictions  do  emerge 
from  the  analysis. 

The  checklist  procedure  certainly  has  one  thing  to  recommend  it:  the 
process  of  scoring  the  design  and  support  features  may  bring  out  serious 
faults. 

Three  objectives  to  checklist  predictions,  however,  are  (1)  the  weights, 
though  statistically  derived  and  "objective"  for  the  system  originally 
studied,  are  seldom  cross-validated  on  other  equipments,  (2)  the  design 
features  scored  tend  to  be  observable  and  primarily  independent--complicated 
internal  features  and  interactions  tend  to  be  ignored,  and  (3)  the  reliability 
of  the  predictions  made,  and  of  the  predictors  themselves,  is  seldom  known. 

For  such  reasons,  it  may  be  well  to  regard  checklist  reviews  as  useful  for  the 
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internal  design  staff,  rather  than  as  satisfactory  quantitative  prediction 
schemes. 

Counting  Methods 

At  the  extremes,  sheer  numbers  can  seem  to  dominate  a  maintenance 
situation.  An  equipment  that  has  50,000  parts  should  be  a  difficult  thing  to 
service.  So  one  indicator  of  fault-locating  difficulty  could  be  the  number  of 
hardware  elements  that  the  technician  has  to  consider. 

Of  course,  much  depends  on  the  way  that  the  parts  are  arranged,  and  on  the 
possibilities  for  block  elimination  of  whole  segments  of  the  equipment. 

Several  projects  have  tried  to  combine  some  notion  of  the  richness  of  test 
indications  with  a  parts  count.  Leuba  (1962),  for  instance,  proposed  a 
measure  in  which  maintainability  varied  directly  with  the  number  of  elements 
in  the  system,  and  the  number  of  symptoms  which  can  be  caused  by  several 
different  elements. 

Sophisticated  counting  techniques  may  yield  quantitative  relationships 
between  repair  time  and  the  counted  elements  which  are  useful  in  projecting 
the  likely  maintenance  load  imposed  by  a  system.  It  must  be  realized, 
however,  that  pure  counting  measures  which  prove  to  be  correlated  with  repair 
time  may,  in  fact,  only  be  indirect  indications  of  system  size,  scope,  and 
complexity.  We  might  equally  expect  measures  such  as  system  weight  or  system 
volume  to  also  provide  significant  correlations.  Thus,  most  attempts  to 
derive  a  counting  measure  incorporate  features  of  system  structure  beyond 
sheer  number. 

Cognitive  Methods 

A  cognitive  approach  to  projecting  maintenance  workload  postulates 
specific  mental  processes  involved  in  troubleshooting  and  seeks  to  identify 
aspects  of  design  which  bear  on  those  processes.  Such  processes  might  include 
perceptual  or  pattern  recognition  systems,  a  memory  component,  as  well  as 
processes  for  inference.  Additionally,  one  may  characterize  various 
strategies  for  troubleshooting  in  terms  of  these  component  cognitive  skills, 
how  they  are  interrelated,  and  when  they  are  used.  Thus,  aspects  of  equipment 
design  may  be  sought  which  impact  these  cognitive  strategies  via  their  effect 
on  underlying  cognitive  processes. 

While  a  model  based  on  cognitive  theory  offers  great  promise,  formulating 
the  mental  processes  involved  will  be  exceedingly  difficult  to  develop  for 
practical  use  in  the  foreseeable  future. 

Complexity  Measures 

It  seems  quite  reasonable  to  look  for  some  way  of  describing  the 
"complexity"  of  a  system  and  then  demonstrating  the  precise  relationship 
between  system  complexity  and  various  aspects  of  maintenance  task  performance 
such  as  mean-time-to-repair  (MTTR) .  Unfortunately,  complexity  is  a  difficult 
concept  to  define  and  quantify.  Some  researchers  have  taken  the  attitude  that 
complexity  is  whatever  affects  maintenance  time,  which  may  be  one  of  the  few 
possible  definitions.  This  definition,  however,  results  in  a  circular 
process,  again  involving  a  search  for  any  factors  which  correlate  with 
maintenance  time.  As  discussed  above,  such  correlational  techniques  do  not 


provide  a  measurement  tool  which  is  sensitive  to  design  alternatives. 
Time-Synthesis  Simulation  Methods 


Psychologists  frequently  break  down  whole  tasks  into  simpler  elements. 
These  subtask  elements  are  then  separately  studied  and  combined  in  various 
ways.  If  the  subtask  performance  parameters  are  defined  probabilistically, 
then  appropriate  distributions  of  overall  performance  values  can  be 
generated.  If  the  global  performance  parameters  agree  well  with  those 
observed  in  the  real  case,  then  the  model  is  said  to  be  validated.  The 
synthesis  can  be  further  validated  if  expected  changes  in  real  performance 
come  from  experimentally  produced  changes  in  the  micro-elements. 

Several  projects  have  employed  time-synthesis  simulation  with  generally 
positive  results  (Rigney,  Cremer,  Towne,  &  Mason,  1966;  Siegel  &  Wolf,  1969; 
Strieb,  Glenn  &  Wherry,  1980). 

The  concept  of  time-synthesis  simulation  is  a  powerful  one.  Parameters 
can  be  varied  easily,  and  hundreds  of  thousands  of  simulated  task  runs  can  be 
quickly  computed,  so  that  the  (model)  effects  of  possible  change  can  be  tried 
out. 


There  are  challenging  technical  problems  in  all  parts  of  time-synthesis 
simulation.  Many  problems  are  encountered  in  setting  the  right  task 
descriptive  level,  in  obtaining  suitable  data  about  human  performance,  and  in 
managing  the  problems  of  task  correlation  and  level  shifting.  Though  some 
complex  behavioral  routines  have  a  straightforward  sequence  of  subtasks,  it  is 
often  difficult  to  synthesize  a  troubleshooting  sequence  that  resembles  human 
performance.  The  technique  described  in  this  paper  is  a  type  of 
time-synthesis  technique. 

An  Analytic  Approach  to  Projecting  Maintenance  Workload 

The  major  problem  in  developing  a  technique  for  assessing  the  impact  of  a 
design  upon  the  maintainer  is  with  projecting  what  particular  actions  are 
likely  to  be  performed  to  isolate  various  faults.  Subsequent  analyses  of 
these  actions,  to  evaluate  performance  time  or  difficulty,  for  example,  are 
manageable  problems  once  the  constituent  actions  are  specified. 

An  ideal  technique  would  project  maintenance  performance  across  a  wide 
range  of  proficiency  and  environmental  levels,  allowing  designers  and  planners 
to  evaluate  the  sensitivity  of  the  design  to  those  variations.  Such  a 
technique  would  reflect  the  variations  in  maintenance  efficiency,  as  well  as 
the  possibly  more  significant  variations  in  error  commission,  error  severity, 
and  error  detection. 

A  more  attainable  approach,  pursued  here,  compares  and  evaluates  designs 
based  on  projections  of  performance  in  a  normal  environment  in  which  tests  are 
performed  correctly  (but  not  necessarily  interpreted  correctly).  Such  a 
capability  may  provide  the  basis  for  extrapolating  to  more  error-prone 
performance  at  a  later  time. 

The  variations  of  possible  performance  are,  of  course,  immense.  At  one 
extreme  is  optimal  performance;  the  strategy  employed  minimizes  the  time 
expected  to  find  and  resolve  a  failure.  At  the  other  extreme  is  a  strategy  in 


which  tests  are  selected  at  random;  no  consideration  of  efficiency  is  made. 

In  between  are  a  vast  array  of  nonoptimal  maintenance  task  sequences  which 
reflect  individual  skills,  training,  and  abilities.  We  have  formulated  eight 
generic  troubleshooting  strategies  in  this  domain,  which,  when  applied  to  a 
specific  representation  of  a  system  design,  generate  troubleshooting  action 
•sequences.  Times  to  perform  these  sequences  are  then  computed  by  retrieving 
and  accumulating  predetermined,  standardized  motion  times  for  the  actions 
involved.  Each  performance  sequence  and  time,  therefore,  reflects  the  total 
impact  of  the  system  design  upon  the  maintainer,  if  he  were  to  follow  the 
particular  strategy.  Moreover,  any  design  change  which  would  affect  the 
maintainer  would  also  affect  the  synthesized  task  sequences  and/or  tne 
performance  times  of  the  constituent  actions. 

System  Representation 

To  represent  a  system  design,  we  require  (1)  a  characterization  of  the 
symptom  information  regarding  the  state  of  the  system,  which  can  be  accessed 
by  the  technician,  (2)  reliability  data,  (3)  data  expressing  the  "cost"  of 
acquiring  that  information,  and  (4)  a  representation  of  the  physical  structure 
of  the  system. 

The  first  two  of  these  can  be  organized  as  a  matrix  as  shown  in  Figure  1. 
The  columns  in  the  body  of  the  matrix  represent  replaceable  units  (RUs)  while 
rows  represent  tests.  Each  cell  entry,  S^j,  expresses  the  consequence  upon 
testi  of  a  failure  in  RUj.  An  entry  of  zero  indicates  no  effect,  i.e., 
test,-  is  unaffected  be  RUj.  A  nonzero  entry  indicates  an  abnormal  symptom. 


R 

:  P  L  / 

ICE/ 

B  L  E 

U  N 

I  T 

TEST 

1 

2 

3 

4 

5 

6 

1 

Sll 

S12 

S13 

•  •  • 

S16 

2 

S21 

3 

S31 

4 

• 

5 

• 

6 

• 

7 

8 

S81 

S86 

1 

R1 

R2 

R3 

R4 

R5 

R6 

Figure  1.  Symptom-malfunction  matrix  with  test  costs  and  unit  reliability. 


The  physical  structure  of  the  system  will  be  represented  as  an  indentured 
assembly  specification  as  shown  in  Figure  2.  All  system  elements  appearing  in 


the  first  (leftmost)  column  are  accessible  to  the  maintainer;  the  time  to 
remove  and  replace  each  is  entered  in  the  last  column.  An  element  appearing 
n  the  second  column  is  accessible  only  by  first  removing  the  element  which 
appears  above  it  in  the  first  column,  and  so  on.  Tests  are  included  in  this 
structural  representation  to  indicate  what  disassembly  must  be  accomplished  to 
initiate  each.  The  test  times  shown  are,  therefore,  the  inherent  times  which 
are  independent  of  preceding  work. 


Figure  2.  Assembly  specifications. 

Fixed  sequences.  Some  portions  of  maintenance  procedures  are  well 
defined — cal ibration  procedures  are  fully  procedural i zed;  some  fault 
localization  procedures  are  prescribed  often  by  the  technical  documentation  or 
are  dictated  by  built-in  test  functions;  and  most  replacement  procedures  are 
predictable  from  knowledge  of  the  system  structure.  While  individual 
technicians  may  differ  in  workspace  and  efficiency  of  performing,  the 
technical  documentation  and  system  design  constrain  the  actions  which  can 
correctly  be  performed. 

The  time  data  for  performing  tests  and  assembly/disassembly  actions  may  be 
based  upon  estimates,  micromotion  analysis,  or  a  mixture  of  these.  Estimates 
would  be  used  when  design  specifications  are  not  detailed,  or  when  highly 
precise  results  are  not  required  or  justified. 

Micromotion  analysis  is  the  synthesis  of  a  defined  task  from  small, 
preanalyzed  motions  (Karger  &  Bayha,  1966).  While  this  approach  yields 
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accurate  results  and  detailed  motion  documentation,  it  requires  considerable 
training  and  application  effort.  An  example  of  a  micromotion  analysis  of 
connecting  a  coax  connector  to  a  receptacle  is  shown  in  Figure  3. 
Fortunately,  a  wide  variety  of  testing,  assembly/disassembly,  and  repair 
operations  have  been  analyzed  and  documented  in  task  time  data  banks. 
Consequently,  a  time  value  for  a  task  may  be  retrieved  from  such  a  catalog, 
rather  than  being  built  up  from  detailed  motion  analysis.  An  automated 
technique,  similar  to  one  now  used  in  industry  (Towne,  1968,  1980),  will  be 
developed  as  part  of  this  research  to  further  facilitate  this  data  retrieval 
process. 
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Figure  3.  Micromotion  analysis  -  attach  coax  connector  to  receptacle. 

Variable  sequences.  Performance  generated  by  individual  technicians  to 
isolate  a  fault  is  difficult  to  predict.  Individual  differences  lead  to  a 
wide  variety  of  approaches,  each  involving  differing  amounts  of  cognitive  and 
manual  labor.  In  addition,  errors  are  often  committed.  Some  of  these  are 
reasoning  faults  which  merely  delay  the  identification  of  the  true  fault. 
Others  lead  the  technician  on  a  long  and  fruitless  search  which  must 
ultimately  be  abandoned  if  the  fault  is  to  be  found.  Other  errors  are  manual 
in  nature,  and  may  be  insignificant,  moderate,  or  catastrophic.  The 
objective,  therefore,  is  to  formulate  a  technique  for  generating  action 
sequences  which  are  typical  of  some  representative  population  of  maintenance 
technicians.  The  time  and  difficulty  of  performing  the  representative 
procedures  may  then  be  determined.  Ideally,  this  process  yields  not  only  a 
mean  (or  expected)  repair  time,  but  also  provides  a  measure  of  likely 
variation. 


We  have  formulated  a  family  of  eight  primitive  troubleshooting  strategies 
(Figure  4)  to  represent  the  possible  approaches  employed  by  individual 
troubleshooters  in  particular  situations.  When  applied  to  a  representation  of 
a  system,  these  strategies  produce  fault  trees  whose  structure  and  performance 
time  cost  are  a  direct  result  of  the  system  design  (as  well  as  the  underlying 
strategy  which  produced  them).  Moreover,  the  fault  trees  are  quite  sensitive 
to  even  minute  design  changes--removing  an  indicator,  changing  the  number  of 
screws  securing  a  module,  or  adding  a  test  point  would  all  be  examples  of 
design  actions  which  would  impact  the  fault  trees. 
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Figure  4.  Eight  generic  fault  isolation  strategies. 

For  each  strategy,  a  particular  rule  is  applied  to  select  each  test.  The 
optimum  strategy,  for  example,  says  to  select  the  test  which  is  likely  to 
return  the  most  information  for  the  time  invested  in  performing  the  test.  The 
symptom-malfunction  matrix  then  indicates  which  system  failures  would  give  a 
normal  indication  and  which  would  cause  an  abnormal  indication  for  that  test. 
The  selection  rule  is  again  applied  to  each  resulting  subset,  and  so  on,  until 
a  complete  fault  tree  is  developed  (Figure  5).  The  time  cost  of  isolating 
each  element  is  then  computed  as  the  sum  of  the  times  of  all  tests  which 
appear  in  the  branch  terminating  at  the  element.  The  measure  of  effectiveness 
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of  a  fault  tree  is  expected  fault  isolation  time,  computed  as: 

E  =  ZR*j  t^ 

where  E  is  Expected  fault  isolation  time 
Ri  is  the  Reliability  of  element  i 
t-f  is  the  time  to  isolate  element  i,  that  is, 
the  sum  of  all  test  times  in  the  branch 
terminating  at  element  i. 

Thus,  in  Figure  5,  the  expected  fault  isolation  time  is  8.6  minutes. 

The  three  variables,  considered  in  various  combinations  by  the  strategies, 
are  test  power  (the  extent  to  which  the  test  is  likely  to  separate  the 
possible  faults  into  subsets),  test  performance  time,  and  element 
reliability.  Strategy  1  considers  all  three  of  these,  and  produces  a  fault 
isolation  procedure  (tree)  which  is  optimal,  i.e.,  the  expected  fault 
isolation  time  is  minimal. 

This  strategy  is  determined  by  computing,  at  each  stage  in  the 
troubleshooting  process,  that  test  which  provides  the  maximum  information  per 
unit  time.  Information  is  computed  according  to  Bayes'  theorem  as  the 
reduction  in  the  total  system  uncertainty,  i.e.: 

Au  =z  Pi  log2  Pi  -  zpr  log 2  p: 

where  Au  =  uncertainty  reduction 

p.  =  probability  of  ith  malfunction  prior  to  test 

pj  =  probability  of  ith  malfunction  after  test 

In  general,  this  algorithm  may  not  yield  a  true  minimum,  as  the  stepwise 
process  does  not  consider  the  characteristics  of  the  fault  areas  discriminated 
at  each  stage.  A  dynamic  programming  formulation  was  implemented  to  compute  a 
true  minimum.  This  process  essentially  "looks  ahead,"  down  each  branch  of  the 
fault  isolation  tree,  and  is  able  to  generate  a  slightly  more  efficient 
strategy.  In  one  application  of  the  Bayesian  process,  the  expected 
troubleshooting  time  for  a  system  was  11.702  minutes,  whereas  the  dynamic 
programming  process  yielded  11.568  minutes.  If  this  close  correspondence 
between  results  holds  up  for  other  systems,  we  v/ill  employ  the  Bayesian 
processor  to  estimate  the  optimum,  as  it  is  a  rapid  computation  compared  to 
the  heavy  computation  load  of  dynamic  programming. 

It  must  be  emphasized  that  the  compute  load  to  generate  the  optimum  used 
here  was  not  considered  by  the  processor  itself,  i.e.,  the  definition  of 
optimality  does  not  embrace  time  invested  in  producing  the  result.  Human 
performers,  on  the  other  hand,  seem  to  be  quite  sensitive  to  the  time  costs 
associated  with  planning  their  performance.  Field  troubleshooters  have  at 
times  been  criticized  for  performing  tests  when  more  planning  and  analysis 
seemed  more  productive.  Whether  or  not  maintainers  tend  to  "under-plan,"  it 
is  important  to  distinguish  between  machine-computed  solutions,  and  those 
developed  in  real  time  by  human  maintainers  who  forego  manual  performance  »:u 
conduct  cognitive  tasks. 

At  the  opposite  extreme  is  a  strategy  in  which  tests  are  selected  at 
random  from  the  set  of  all  tests  which  can  offer  any  information  about  the 
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status  of  the  system.  The  random  strategy  provides  an  upper  limit  on  rational 
troubleshooting  time. 

Between  the  optimal  strategy  and  the  random  strategy  (on  the  dimension  of 
effectiveness)  lie  six  rational,  suboptimal  approaches,  each  of  which 
considers  one  or  two  of  the  three  variables  used  by  the  optimum  strategy.  A 
brief  summary  of  all  eight  strategies  follows. 

1.  Optimum  test  selection.  Tests  are  selected  to  minimize  total  expected 
fault  isolation  time.  This  strategy  considers  the  time  costs  of  the 
tests,  the  power  of  the  tests,  and  the  relative  reliabilities  of  the 
system  elements. 

2.  Element  half-splitting,  per  unit  time.  Tests  are  selected  to  best  split 
the  suspected  elements  into  two  subsets  of  equal  size,  per  unit  time. 

This  is  similar  to  strategy  1  with  initial  element  reliabilities  ignored. 

3.  Briefest  test  solution.  The  briefest  test  which  can  provide  any 
information  is  selected  at  each  stage.  Only  time  cost  is  considered  in 
the  selection. 

4.  Half-splitting  by  reliability.  Tests  are  selected  to  best  split  the 
suspected  elements  into  two  subsets  of  equal  failure  probability.  This  is 
similar  to  the  strategy  with  test  time  cost  ignored. 

5.  Half-splitting  by  element.  Tests  are  selected  to  best  split  the  suspected 
elements  into  two  subsets  of  equal  size.  This  is  equivalent  to  strategy  2 
with  test  time  cost  ignored. 

6.  Check  least  reliable  element,  per  unit  time.  Tests  are  selected  to 
monitor  the  greatest  probability  of  failure  per  unit  time.  Test  time  cost 
and  element  reliability  are  considered. 

7.  Check  least  reliability  element.  Tests  are  selected  to  check  the  least 
reliable  elements  first.  Only  reliability  is  considered  in  the  selections. 

8.  Random  test  selection.  Tests  are  selected  at  random  (no  repeating) 
without  regard  to  test  time  cost,  test  power,  or  reliabilities. 

These  eight  strategies  were  applied  to  a  microcomputer  system  consisting 
of  mainframe,  video  terminal,  hardcopy  printer,  and  disk  drive  unit  (Figure 
6).  The  representation  of  the  system  is  shown  in  Figure  7. 

Experimentation  is  now  underway  to  determine  how  actual  troubleshooting 
action  sequences  compare  to  these  baseline  strategies.  A  comparison  of  the 
eight  strategies,  summarized  in  Figure  8,  is  interesting  in  its  own  right. 

The  simple  strategy  of  performing  the  briefest  productive  test  (strategy  3) 
yielded  an  expected  fault  isolation  time  of  13.5  minutes,  surprisingly  close 
to  the  11.7  minute  optimum.  Strategy  2,  which  uses  test  power  and  test  cost, 
yielded  13.2  minutes  expected  fault  isolation  time,  indicating  that  initial 
reliability  contributed  little  to  the  solution.  The  classical  half-splitting 
strategy  (perform  a  test  to  split  the  system  in  two)  yields  21.3  minutes, 
whereas  half-splitting  into  two  equally  reliable  subsets  (strategy  4)  requires 
less  time  at  16.8  minutes. 


The  two  strategies  which  emphasize  checking  unreliable  elements  perform 
poorly,  at  over  forty  minutes.  These  results  are  surprisingly  close  to  random 
test  selection  (Figure  8),  which  yields  a  mean  expected  repair  time  of  49.7 
minutes  (N=800). 

Examination  of  Figure  8,  reveals  that  the  rank-order  of  fault  isolation 
times  for  individual  faults  are  relatively  consistent  across  strategies. 

Those  approaches  which  ignore  test  time  cause  the  greatest  departures  from 
this  tendency,  since  they  may  call  for  performing  lengthy  tests  to  check  just 
a  few  unreliable  elements. 
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Figure  8.  Element  isolation  times  (minutes)  for  eight  generic  strategies. 

The  results  of  this  one  analysis  certainly  do  not  constitute  a  basis  for 
generalization.  Since  the  optimum  strategy  provides  a  true  baseline  of  expert 
performance,  it  may  prove  to  correlate  best  with  observed  maintenance 
activity,  across  different  systems.  If  maintainers  are  generally  parsimonious 
with  time  but  not  particularly  prone  to  consider  test  power,  then  we  may  find 
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actual  maintenance  performance  resembles  that  of  strategy  3.  If,  instead, 
maintainers  focus  their  attention  on  unreliable  elements,  then  we  might  expect 
performance  more  like  strategy  7.  And,  if  maintainers  switch  among 
time-dominant,  reliability-dominant,  and  test  power-dominant  strategies,  we 
might  expect  some  function  of  strategies  3,  5,  and  7  to  provide  a  projection 
of  maintenance  workload.  For  example,  if  there  is  a  tendency  to  select  quick 
and  easy  tests  early  in  a  problem,  and  later  shift  to  an  enumerative  search 
process  as  the  possible  faults  emerge,  we  may  employ  strategies  3  and  7  to 
project  the  performance.  Experimentation  is  needed  to  determine  if  such 
shifting  strategy  techniques  are  used  by  maintainers,  and  if  so,  to  determine 
when  and  under  what  conditions  in  a  fault  isolation  task  such  shifts  will 
occur. 

The  most  intriguing  result  of  this  one  application  is  that  the  fault 
isolation  performances  and  times  were  relatively  constant  across  the 
time-dominant  strategies,  and  were  relatively  constant  at  a  higher  level 
across  the  two  reliability-dominant  strategies.  This  suggests  the  interesting 
and  very  tentative  hypothesis  that  the  work  required  to  isolate  a  particular 
fault  may  be  highly  determined  by  the  design  and  less  sensitive  to  individual 
differences  of  isolation  method.  Further  application  and  experimentation  are 
needed  to  test  these  early  impressions. 
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Safety  in  today's  world  has  taken  on  new  meaning.  To  the  professional, 
safety  is  no  longer  the  simplistic  "freedom  from  hazard  for  man."  Instead,  a 
new  and  much  broader  definition  has  come  into  use.  Now,  safety  means  "freedom 
from  the  people/equipment/material /environment  interactions  that  result  in 
Injury  to  personnel,  damage  to  the  system,  time  loss  or  any  downgrading  of  the 
mission  objectives."  This  broadened  scope  of  safety  actually  improves  the 
overall  protection  of  the  individual  when  compared  to  the  previous  one. 

The  improved  definition  does  not  restrict  the  safety  effort  to  man  alone, 
rather  it  embodies  the  system  within  which  man  operates.  When  one  considers 
this  system,  any  loss  of  any  attributes  of  the  system  becomes  the  main  issue. 
As  such,  loss  control  then  becomes  the  goal  of  safety,  whether  it  is  the 
preservation  of  human/equipment/material  or  environmental  assets. 

Another  very  important  point  to  bear  in  mind  about  preventing  loss  is  that 
losses  (or  "accidents")  are  not  necessarily  time  based,  i.e.,  they  do  not 
always  suddenly  occur  as  in  a  plane  crash  or  a  forest  fire.  Long-term 
exposure  to  a  variety  of  energy  sources  may  lead  to  unacceptable  losses.  On 
the  human  side,  consider  the  health  hazards  of  exposure  to  cotton  fibers,  coal 
dust,  asbestos,  vibration,  high  noise  levels,  etc.  The  courts  have  concluded 
that  these  injuries  are  "accidents"  if  there  was  sufficient  knowledge 
available  on  the  long-term  effects.  Consider  the  slow  degradation  of  our 
planet's  ecology  and  natural  resources  as  examples  of  long-term  losses  that 
need  to  be  controlled. 

Losses  occur  as  a  result  of  some  sort  of  energy,  be  it  mechanical, 
chemical,  electrical,  or  whatever.  To  minimize  losses,  one  has  to  identify, 
analyze,  and  control  these  energies.  For  Instance,  consider  fire.  The  three 
things  that  generally  lead  to  a  fire  are  fuel,  oxygen,  and  an  ignition 
source.  Removing  any  one  will  stop  the  fire  from  starting  or,  if  started,  put 
it  out.  Removal  of  the  risk  of  fire  Implies  preplanning  and  that  is  precisely 
what  loss  control  Is  all  about,  thinking  ahead.  There  will  always  be 
situations  where  the  risk  cannot  be  completely  removed.  In  these  situations, 
a  conscious  decision  has  to  be  made  to  accept  this  risk.  In  the  case  of  fire. 
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for  example,  risk  acceptance  may  hinge  on  adequate  fire  extinguishing 
facilities  being  available.  Corrosion  is  very  similar  to  fire.  After  all, 
v/hat  is  fire  but  a  very  rapid  corrosion.  Loss  control  due  to  corrosion  can  be 
tackled  in  a  similar  fashion. 

There  are  two  basic  premises  in  loss  control:  (a)  the  incidents  that 
downgrade  the  system's  capability  are  caused;  they  do  not  just  happen,  and  (b) 
these  causes  can  be  determined  and  controlled. 

The  four  sources  for  these  causes  are  given  in  the  definition.  They  are: 
(1)  people,  (?)  equipment,  (3)  material,  and  (4)  environment.  Seldom  does  any 
one  act  alone  to  cause  an  incident. 

People  need  to  have  proper  training  and  leadership  in  order  to  be  able  to 
do  their  job  with  pride.  They  also  need  good  equipment  that  is  designed 
properly,  available  with  the  proper  tools  to  repair,  operate,  and  maintain 
it.  The  material  that  has  to  be  worked  with  may  be  toxic  or  otherwise 
dangerous,  so  proper  safeguards  must  be  taught,  available  and  utilized. 

Lastly,  the  environment  must  be  benign  or  the  workers  must  be  protected  from 
excessive  noise,  wind,  moisture,  heat,  cold,  etc. 

The  above  examples  are  but  a  few  of  the  myriad  of  possible  variations 
available  in  each  broad  causal  category.  For  example,  people  vary  in 
intelligence,  perceptual  ability,  physiology,  moods,  emotions,  etc.  After  all 
is  considered,  the  permutations  and  combinations  possible  are  astronomical. 

Yet  any  one  of  these  is  the  rare  occurrence  that  may  result  in  a  catastrophic 
loss  of  life  or  property.  In  some  cases,  we  may  even  be  talking  about  the 
eventual  end  of  life  on  our  planet.  The  goal  of  loss  control  professionals  is 
to  attempt  to  preconceive  the  worst  of  the  consequences  and  then  find  some 
means  of  controlling  or  outright  avoiding  them.  Their  success  is  measured  in 
lives  and  dollars,  yet  it  is  difficult  to  quantify  that  which  has  never  been. 
As  more  and  more  systems  have  loss  control  theory  applied  to  their 
development,  comparisons  will  be  able  to  be  made  to  illustrate  these  savings 
in  a  more  understandable  form. 


Techniques  for  Hazard  Identification 

The  greatest  difficulty  with  loss  control  application  is  the 
identification  of  hazards.  You  cannot  control  or  eliminate  what  you  do  not 
recognize.  Consequently,  many  different  types  of  checklists  have  been 
developed  to  assist  in  the  identification  of  possible  energy  sources  (see 
Tables  1  and  2).  Checklists  are  only  intended  to  provide  a  starting  point 
since  most  systems  incorporate  unique  sources  of  energy  problems. 

When  using  any  checklist,  it  is  important  to  realize  failures  may  occur  at 
different  points  in  the  development  of  a  new  system.  That  is,  during  the 
engineering  phase,  common  failures  may  occur  due  to  the  design  or  the 
construction  of  the  system.  Design  failures  may  be  further  subdivided  to 
include  functional  deficiencies  such  as  undetected  hazards  or  inadequate 
controls,  and  realization  faults  found  after  the  design  is  complete  such  as 
inadequate  materials,  operational  deficiencies,  channel  dependency,  etc. 
Construction  failures  may  be  due  to  either  manufacturing  or  installation. 

These  types  of  failures  generally  result  from  inadequate  inspections,  quality 
control,  testing,  standards,  etc. 


During  the  operational  phase,  failures  nay  be  caused  by  either  procedural 
or  environmental  considerations.  Procedural  failures  nay  result  from  either 
maintenance  or  normal  operations.  Maintenance  errors  generally  result  from 
lack  of  precision  in  repairs,  calibrations,  testing,  or  procedures,  whereas 
operations  failures  result  from  human  error  (either  operator  or  supervisor), 
inadequate  procedures,  or  communication  breakdowns. 

Environmental  failures  relate  more  directly  to  energy  sources  and  are  thus 
easier  to  prepare  against.  There  are  the  normal  energy  extremes  encountered 
such  as  shock,  temperature,  pressure,  vibration,  humidity,  etc.,  as  well  as 
the  energy  extremes  from  external  events  that  may  be  encountered  such  as  fire, 
flood,  explosions,  earthquakes,  radiation,  etc. 

Checklists  that  are  used  and  annotated  can  be  filed  and  represent  the 
nucleus  of  a  corporate  mind  relative  to  the  developing  systems  as  well  as 
similar  future  systems.  They  also  provide  an  invaluable  source  to  return  to 
when  unexpected  failures  occur,  thereby  becoming  a  very  expensive 
lesson-learned  file.  Any  failure  of  any  system  should  be  recorded  and  used  to 
prevent  similar  failures  in  the  future  on  new  systems. 

Table  1 


Energy  Source  Checklist 


1 . 

Fuel  s 

12. 

Electrical  Generators 

2. 

Propellants 

13. 

Electromagnetic  Radiation 

3. 

Initiators 

14. 

Radioactive  Energy  Sources 

4. 

Explosive  Charges 

15. 

Falling  Objects 

5. 

Charged  Electrical  Capacitors 

16. 

Catapulted  Objects 

6. 

Storage  Batteries 

17. 

Heating  Devices 

7. 

Static  Electrical  Charges 

18. 

Pumps,  Blowers,  Fans 

a. 

Pressure  Containers 

19. 

Rotating  Machinery 

9. 

Spring-loaded  Devices 

20. 

Actuating  Devices 

10. 

Suspension  Systems 

21. 

Nuclear 

11. 

Gas  Generators 

22. 

Cryogenics 

Table  2 


General  Hazard  Source  Checklist 


1. 

Acceleration 

11.  Oxidation 

2. 

Contamination 

12.  Pressure 

3. 

Corrosion 

High 

4. 

Chemical  Dissociation 

Low 

5. 

Electrical 

Rapid  Changes 

Shock 

13.  Radiation 

Thermal 

Thermal 

Inadvertent  Activation 

Electromagnetic 

Power  Source  Failure 

Ionizing 

Electromagnetic  Radiation 

111  traviolet 

6. 

Explosion 

14.  Chemical  Replacement 

7. 

Fire 

15.  Shock  (Mechanical) 

8. 

Heat  and  Temperature 

16.  Stress  Concentrations 

High  Temperature 

17.  Stress  Reversals 

Low  Temperature 

18.  Structural  Damage  or  Failure 

Temperature  Variations 

19.  Toxicity 

9. 

Leakage 

20.  Vibration  and  Noise 

10. 

Moisture 

21.  Weather  and  Environment 

High  Humidity 
Low  Humidity 


When  a  new  system  is  first  conceived  is  the  time  to  begin  the  first  hazard 
analysis.  This  initial  hazard  analysis  is  called  the  Preliminary  Hazard 
Analysis  (PHA)  and  should  represent  an  attempt  to  define  all  of  the  energy 
sources  inherent  v/i thin  the  proposed  design.  Knowledge  of  these  hazards  can 
help  the  engineering  staff  in  determining  design  characteristics  that  help  to 
minimize  potential  hazards.  As  the  system  matures  and  becomes  more  defined  by 
‘the  design,  the  PHA  must  be  updated  to  encompass  the  new  definitions.  An 
effective  PHA  v/i  11  have  its  impact  felt  throughout  the  life-cycle  of  any 
product.  An  example  format  for  performing  a  PHA  is  given  in  Figure  1.  The 
columnar  format  is  used  because  a  given  portion  of  the  proposed  system  may 
have  more  than  one  failure  mode,  with  each  mode  having  a  different  probability 
of  occurrence  with  varying  effects  on  the  overall  system.  These  differing 
effects  may  in  turn  lead  to  different  degrees  of  hazard  severity.  Obviously, 
different  failure  modes  and  effects  require  different  types  of  controls  to 
eliminate  or  reduce  the  probability  of  a  hazard  occurrence. 

The  PHA  is  the  first  hazard  analysis  performed,  and  as  such,  is  very  broad 
and  gross.  Other  hazard  analyses  are  performed  at  different  parts  of  the 
life-cycle  as  more  detailed  information  becomes  available.  If  the  PHA  was 
properly  done,  these  other  hazard  analyses  will  be  considerably  easier.  Other 
hazard  analyses  consist  of  a  close  look  at  each  subsystem  and  how  it  marries 
to  the  whole,  the  effects  of  radiation,  the  effects  of  electromagnetic  pulses, 
and  other  hazards  associated  with  the  normal  operation  and  maintenance  of  the 
system.  Although  all  of  these  hazard  analyses  are  important,  the  Maintenance 
Hazard  Analysis  (MHA)  will  be  used  as  an  example  since  it  is  of  particular 
interest  to  the  reader.  As  can  be  seen  from  Figure  2,  the  MHA  is  similar  to 
the  PHA  insofar  as  the  columnar  format  is  concerned.  Note  that  a  MHA  is 
performed  for  each  subsystem.  This  assures  that  all  required  maintenance 
hazards  can  be  identified  and  accounted  for. 

The  MHA  is  started  early  in  the  validation  phase  of  the  life-cycle  prior 
to  the  first  design  review.  It  is  updated  as  system  design  solidifies  and 
should  be  reviewed  for  each  modification,  redesign,  or  engineering  change  that 
occurs  after  its  completion. 

The  analysis  uses  information  from  engineering  design  data,  descriptive 
data  of  support  and  test  equipment,  and  actual  hardware  inspection.  For 
example,  the  analyst  must  constantly  consider  the  rolling  and  pitching  deck 
environment  of  ships  when  assessing  hazards  associated  with  the  maintenance  of 
a  waterborne  system.  Human  factors  must  be  considered  with  regard  to 
anthropometry,  strength,  educational  levels,  cognitive  loadings,  etc.  In 
addition,  maintenance  tasks  must  consider  lifting  requirements,  physical 
support,  exposure  to  high  voltages,  release  of  pressures,  fluids  and/or  gases, 
exposure  to  microwaves  or  X  rays,  disposal  of  toxic  substances,  and  general 
interference  by  or  with  flexible  or  fixed  cables,  piping,  or  similar  equipment 
subject  to  damage  by  abrasion  or  impact. 

The  MHA  form  given  in  Figure  2  is  completed  by  carrying  out  the  following 
steps: 

Step  1.  General  -  The  title  block  data  identifies  the  system  and 
subsystem  being  analyzed,  maintenance  level  (organization  or  intermediate) 
data,  and  other  self-explanatory  information. 


ile  format  for  PHA 


fomat  for  MHA 


Step  2.  Column  1  -  Equipment  -  This  column  divides  the  analysis  into 
sections.  The  listing  should  be  ordered  by  the  system  Work  Breakdown 
Structure.  The  entry  should  show  the  major  assembly,  such  as  wheel  breaks,  in 
the  landing  gear  subsystem  or  elevators  in  the  flight  control  subsystem,  on 
which  the  maintenance  is  performed. 

Step  3.  Column  2  -  Maintenance  Type  -  This  entry  identifies  the  general 
maintenance  type  being  analyzed.  Entries  may  include  such  types  as  preventive 
maintenance,  corrective  maintenance,  fault  isolation,  etc. 

Step  4.  Column  3  -  Maintenance  Function  -  In  this  column,  define  the 
single  function  being  considered  Tuch  as  lubricate,  adjust,  calibrate,  test, 
isolate  fault,  or  remove  or  replace  an  identified  component. 

Step  5.  Column  4  -  Hazard  -  The  hazard  associated  with  the  function  and 
tasks  identified  by  columns  2  and  3  is  identified  here.  Typical  entries 
include  high  voltages,  moving  parts,  toxic  substances,  inadequate  support,  etc. 

Step  6.  Column  5  -  Hazard  Classification  -  Hazards  can  be  classified 
according  to  their  severity  and  probability  of  occurrence.  For  example, 

Figure  3  shows  a  detailed  breakdown  of  the  most  severe  category  (i.e., 
category  I)  utilized  by  the  Department  of  Defense.  As  can  be  seen,  severity 
is  based  on  the  number  of  lives  or  type  of  equipment  at  risk.  Category  II 
hazards  are  considered  "critical"  and  may  lead  to  severe  injury  or  illness 
and/or  major  damage  to  a  system.  Category  III  hazards  are  "marginal" 
involving  minor  injury  or  illness  and/or  minor  damage  to  a  system.  Category 
IV  hazards  are  "negligible"  meaning  there  is  no  risk  of  injury,  illness  or 
damage.  Obviously,  each  level  of  hazard  except  category  IV  has  an 
unacceptable  rate  of  occurrence  at  which  point  measures  must  be  taken  to 
prevent  that  occurrence.  Figure  4  shows  the  areas  of  acceptable  risk  based  on 
probability  of  occurrence  for  each  hazard  category. 

Step  7.  Column  6  -  Safety  Features  or  Recommendations  -  The  safety 
features  that  have  been  incorporated  in  the  equipment  design  or  maintenance 
plans  and  facilities  are  listed  in  this  column.  Add  any  recommended  controls 
to  prevent  an  accident  if  already-incorporated  safety  features  are  deemed 
inadequate. 

Step  8.  Column  7  -  Remarks  -  The  remarks  column  identifies  the  category  I 
and  II  hazards  in  column  5  that  are  not  eliminated  by  safety  features.  This 
is  where  recommended  corrective  action  such  as  design  changes,  safety  or 
warning  devices,  warnings  signs  and  personnel  training  are  made. 

The  MHA  is  most  often  used  to  develop  requirements  for  cautions,  warnings, 
and  emergency  procedures  for  inclusion  into  maintenance  manuals  and  plans. 


AD-A168  934 
UNCLASSIFIED 


PROCEEDINGS  OF  A  CONFERENCE  ON  A  COMPREHENSIVE 
TECHNICAL  PEVIEU  OF  HUMAN  CU>  NAVAL  AIR  DEVELOPMENT 
CENTER  NARMINSTER  PA  D  1C  MCBRIDE  ET  AL  11  MAR  82 
S8I-AD-F630  624  F/G  5/5 


2/3 


NL 


CATEGORY  I  LEGEND 


CATEGORY 

LIVES 

SMALL  SYSTEM 
(AIRCRAFT) 

LARGE  SYSTEM 
(SHIPS) 

I  A 

1  -  5 

Fighter/Attack 

— 

1  B 
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I  C 
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Figure  3.  Hazard  severity. 
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Figure  4.  Hazard  probability. 
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A  basic  concern  of  the  human  factors  specialist  working  in  safety  is  the 
ability  to  assess  the  impact  of  operator  error  within  the  system  being 
Investigated.  The  techniques  to  be  described  here  allow  for  the  Inclusion  of 
human  error. 

In  order  to  work  with  human  error,  one  must  first  define  what  types  of 
human  error  are  being  considered.  If  you  consider  human  error  as  a  behavior, 
then  it  follows  that  errors  can  result  from  intentional,  unintentional  or 
omitted  acts.  In  any  case,  the  errors  may  be  caused  by  errors  in  the 
Inputting,  mediation  or  outputting  of  information.  Errors  due  to  input 
consist  of  reading  dials  or  scales,  labels,  etc.  Confusion  of  instructions  or 
difficulty  of  interpretation  are  considered  input  problems.  Mediating  errors 
result  from  such  things  as  failures  in  Identification  or  recognition.  Output 
errors  are  response  oriented  such  as  movement  of  levers,  switches,  oral  or 
written  responses,  etc. 

The  point  of  breaking  down  the  possible  areas  where  human  errors  may  occur 
is  to  illustrate  that,  of  the  possible  nine  combinations  between  the  acts  and 
the  processes,  each  one  has  a  distinct  probability  of  occurrence.  The  problem 
is  to  find  out  what  that  probability  is  and  to  determine  the  acceptability  of 
the  associated  risk. 

There  is  a  method  developed  by  Sandia  Corporation  known  as  the  Technique 
for  Human  Error  Rate  Prediction  (THERP)  that  can  be  useful  in  making  these 
decisions.  THERP  presumes  knowledge  of  basic  error  rates  (BER).  Table  3  is 
an  example  of  some  basic  error  rates  for  humans.  Although  there  is  a  large 
list  of  such  error  rates  available,  the  variability  between  operators  and 
working  conditions  require  caution  In  its  use.  Researchers  are  working  to 
improve  both  the  quantity  and  quality  of  such  data. 

The  THERP  process  begins  with  the  analyst  picking  a  specific  high  cost 
failure  with  human  error  in  the  chain  of  antecedent  events.  Then,  the  BER  for 
each  possible  operator  error  Is  assigned  and  the  total  probability  of  failure 
is  calculated.  If  the  probability  of  occurrence  is  too  high  to  accept  the 
risk,  the  analyst  then  goes  back  to  look  at  where  improvements  can  be  made  to 
reduce  human  error  rates,  thereby  reducing  the  overall  failure  rate.  This 
procedure  can  be  repeated  as  often  as  necessary  until  the  risk  is  acceptable. 

Where  does  one  find  those  critical  areas  where  human  errors  can  contribute 
to  the  overall  catastrophic  failure?  PHAs  and  the  associated  columnar  type 
analyses  deal  more  with  hardware  than  with  human  error.  One  of  the  most 
useful  techniques  available  to  an  analyst  that  can  Incorporate  human  error  is 
the  Fault  Tree  Analysis  (FTA) .  The  principles  of  FTA  have  been  around  for 
centuries  In  the  form  of  logic  trees  and,  as  such,  are  actually  representative 
of  symbolic  logic  diagrams. 

Fault  trees  are  useful  to  a  point,  beyond  which  they  can  be 
counterproductive.  A  primary  rule  to  follow  concerning  fault  trees  is  that 
they  should  only  be  done  on  specific  undesired  events  that  have  been 
identified  by  other  hazard  analyses.  The  reason  for  this  is  that  properly 
performed  FTAs  are  very  time-consuming  (hence,  expensive)  and  quite  often 
become  too  large  to  be  meaningful.  Another  associated  problem  is  that  the 


Representative  Human  Error  Rates* 


Task  Element 

Action 

Object 

Error 

BER** 

Observe 

Chart 

Inappropriate  switch  actuation 

1128 

Read 

Gauge 

Incorrectly  read 

5000 

Read 

Instructions 

Procedural  error 

64500 

Connect 

Hose 

Improperly  connected 

4700 

Torque 

Fluid  lines 

Incorrectly  torqued 

104 

Tighten 

Nuts,  bolts 

Not  tightened 

4800 

Install 

Nuts,  bolts 

Not  installed 

600 

Install 

0-ring 

Improperly  installed 

66700 

Sol der 

Connectors 

Improper  solder  joint 

6460 

Assemble 

Connector 

Bent  pins 

1500 

Assemble 

Connectors 

Omitted  parts 

1000 

Close 

Valve 

Not  closed  properly 

1800 

Adjust 

Mechanical  linkage 

Improper  adjustment 

16700 

Install 

Line  orifice 

Wrong  size  installed 

5000 

Machine 

Valve  port 

Wrong  size  drilled  and  tapped 

2083 

(From  NSC  Rpt  2M57009  hy  J.  L.  Recht) 

♦These  data  should  not  be  used  for  computational  purposes  without  additional 
background  information  -  specifically,  under  what  conditions  these  rates  can 
be  expected  to  be  valid  and  the  probable  error  in  each  rate. 

♦♦Basic  error  rate  (errors  per  million  operations). 


final  determination  concerning  the  undesired  event’s  occurrence  is  only  as 
realistic  as  the  assumptions  and  probabilities  assigned  within  the  logic  tree 
Itself. 


A  FTA  is  a  deductive  analytical  means  to  Identify  all  failure  modes 
contributing  to  the  potential  occurrence  of  a  given  top  undesirable  event.  It 
displays  all  the  necessary  and  sufficient  failure  modes  which  cause  the  top 
event.  The  fault  tree  analysis  can  be  completed  in  either  a  qualitative  or 
quantitative  form.  Every  analysis  begins  qualitatively  and  is  most  valuable 
at  this  stage.  This  Is  where  hazards  which  might  otherwise  have  been 
overlooked  are  recognized  and  can  be  addressed. 

Quantification  of  a  FTA  is  simply  a  matter  of  assigning  probability  levels 
to  all  the  basic  events.  These  probability  levels,  in  turn,  are  used  to 
calculate  the  total  probability  that  the  undesired  top  event  may  occur  (as 
will  be  shown  later). 

Fault  trees  are  constructed  using  combinations  of  the  symbols  given  in 
Figure  5.  These  are  not  exhaustive  but  represent  the  most  commonly  used 
symbols  that  allow  for  ease  of  quantification.  Figure  6  is  an  example  of  a 
simple  fault  tree  showing  the  various  levels  of  analysis  using  different 
gates.  Each  alpha  character  represents  a  level  or  segment  of  analysis.  Each 
event  within  a  fault  tree  occurs  in  either  of  two  states,  i.e.,  failed  or  not 
failed,  and  each  condition  has  a  probability  associated  with  it  such  that: 

P(S)  +  P(F)  =  1 

Where  P(S)  is  the  probability  of  success  and  P(F)  is  the  probability  of 
failure. 

Each  level  of  a  fault  tree  will  have  a  probability  of  success  or  failure 
associated  with  it.  For  example,  in  Figure  6,  Event  E2  has  a  probability  of 
failure  based  on  the  failure  rates  of  E21  or  E22  or  E23  or  E24,  v/hereas  Event 
D2  has  a  probability  of  failure  based  on  tTie  failure  rates  of  Events  El  and  E2. 

To  further  illustrate  the  analysis  technique  a  simplistic  example  will  be 
developed.  Assume  a  top  undesirable  event  of  a  hot  start  on  a  jet  engine  and 
the  simplified  logic  tree  as  depicted  in  Figure  7.  Each  event  within  the 
logic  tree  has  a  specific  meaning.  Table  4  identifies  these  various 
meanings.  Note  that  events  Cl  1  and  Cl 3  are  the  same  as  events  C21  and  B22 
respectively. 

The  problem  is  to  determine  the  probability  of  a  hot  start  occurring. 

Table  5  gives  the  assumed  probabilities  for  each  of  the  events.  In  the  real 
world,  these  probabilities  would  come  from  research  data  and  experience. 

Where  nothing  is  available,  an  educated  estimate  is  used.  If  the  event 
without  a  quantitative  probability  is  critical  to  the  survival  of  the  system, 
a  collective  estimate  from  several  experienced  professionals  should  be  used. 
Although  this  technique  may  seem  less  than  scientific,  one  should  bear  in  mind 
that  in  the  absence  of  an  alternative,  anything  is  better  than  nothing.  This 
technique  then  allows  the  analyst  to  change  probability  levels  to  adjust  the 
probability  of  the  outcome  until  an  acceptable  level  is  reached  (note  the 
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Table  4 


Descriptive  Meanings  for  Figure  7 

A:  Hot  start  on  jet  engine 

All:  Wrong  fuel 
A12:  Too  much  fuel  to  engine 
A13:  Residual  fuel  in  engine 
Bll:  Human  error,  wrong  fuel  truck 
Cl  1  =  C21 :  Human  error,  poor  pre/post  flight 
Cl 2:  Wind  from  rear 
Cl 3  =  B22:  Blocked  fuel  drain 
C22:  Quick  turnaround  required 


Table  5 

Given  Probabilities  for  Events  Given  in  Figure  7 


similarity  to  the  THERP  process).  Once  the  probability  levels  have  been 
decided,  the  engineers  can  decide  to  reach  the  proposed  reliability  levels  or 
alter  procedures,  etc.,  to  reach  the  desired  end  point. 

To  calculate  the  overall  probability  of  Event  T,  it  is  useful  to  modify 
our  logic  tree  as  can  be  seen  in  Figure  8.  This  is  done  strictly  to  allow  for 
'easier  algebraic  manipulation. 

The  process  for  determining  Event  T' s  probability  begins  from  the  bottom 
of  the  logic  tree  and  works  upward.  Each  branch  is  done  separately;  that  is, 
first  we  calculate  Y2  then  Yu  and  X2  then  X-| .  Once  we  have 
calculated  the  probability  of  both  Y,  and  X,  then  we  can  calculate  the 
probability  of  Event  T. 

To  calculate  an  event  governed  by  an  'OR'  gate,  the  following  formula  is 
used: 

P0  -  1  -  II"  i=l  (1  -  Qi) 


Where  (jj  is  the  probability  of  the  I**1  causal  event  and  n  is  the 
number  of  parallel  branches.  The  symboT  II  is  a  product  of  terms  symbol.  For 
events  governed  by  an  'AND'  gate,  the  probability  of  occurrence  is  given  by: 

PA  ■  Iin  i=l  Qi 

Substituting  the  given  probabilities  from  Figure  8  into  the  appropriate 
formula,  event  Y2  =  .4,  Y-|  =  .002,  X2  =  1.25  X  10-5  and  Xl  =  1.25  X 
10-8.  Event  T  can  now  be  calculated  using  the  P0  formula. 

Event  T  =  1  -  (1  -  .002) (1  -  .005) (1  -  1.25  X  10“8) 

Yi  A  Xt 

So  Event  T  =  .007 

This  shows  that  the  probability  of  a  hot  start  is  approximately  1  out  of 
100  (actually  7  out  of  1000). 

If  this  frequency  of  occurrence  Is  too  high,  then  the  next  thing  to 
determine  is  where  the  most  payoff  can  be  made  altering  probability  levels. 
There  are  several  techniques  to  determine  this.  All  the  techniques  basically 
do  the  same  thing,  i.e.,  determine  the  minimal  cut  sets  within  the  logic 
system. 

A  minimal  cut  set  can  be  defined  as  the  smallest  group  of  basic  end  events 
whose  collective  occurrence  assures  the  occurrence  of  the  top  event.  The 
basic  end  events  may  be  hardware  failures  or  human  errors.  A  minimal  cut  set 
may  not  contain  another  cut  set  since  that  would  imply  that  the  former  was  not 
minimal.  The  number  of  events  in  a  minimal  cut  set  determines  the  number  of 
failure  points  In  the  system.  For  example,  a  minimal  cut  set  of  one  is  a 
single  point  failure,  two  a  dual  point  and  so  forth. 


The  first  technique  to  determine  minimal  cut  sets  will  be  the  use  of 
symbolic  logic  and  Boolean  Algebra.  Table  6  presents  a  list  of  Boolean 
Algebra  postulates.  When  writing  a  Boolean  expression,  an  'OR'  gate  is 
additive  (note  the  +  sign  in  the  'OR'  gate  in  Figure  8)  and  an  'AND'  gate  is 
multiplicative  (note  the  o  in  the  'AND'  gate  in  Figure  8).  Using  the  modified 
fault  tree  in  Figure  8,  the  Boolean  expression  for  the  logic  system  will  be 
devel oped. 

T  =  X]  +  A  +  Yi 
Where  Xi  =  E(X2) 
and  X2  =  BCD 
so  X]  =  E(BCD)  or  EBCD 
also  Y-|  =  Y2 ( D ) 

Where  Y2  =  B  +  F 
so  Y]  =  (B  +  F)D  or  BD  +  FD 
Therefore  T  =  EBCD  +  A  +  BD  +  FD 

When  working  with  Boolean  Algebra  it  is  useful  to  apply  Rule  11  from  Table 
6  and  keep  alphabetical  order. 

So  T  =  BCDE  +  A  +  BD  +  DF 

Mow  factoring  BD  we  get: 

T  =  BD(CE  +1)  +  A  +  DF 
and  CE  +1=1  (Rule  5) 
so  T  =  BD  +  A  +  DF  or  T  =  A  +  BD  +  DF 

These  represent  the  furthest  reduction  possible  and  as  such  become  the 
minimal  cut  sets  for  this  logic  tree.  Events  C  and  E  are  not  considered 
because  before  conditions  to  cause  them  have  occurred,  events  B  and  D  have 
already  occurred.  Remember  that  BCDE  includes  BD  so  it  is  not  a  minimal  cut 
set. 

Another  method  to  determine  cut  sets  is  to  use  a  chart.  Table  7  shows  the 
process.  Events  under  'AND'  gates  are  placed  horizontally  and  those  governed 
by  'OR'  gates  are  placed  vertically  until  all  end  events  have  been  placed  into 
the  chart.  Once  this  chart  has  been  completed,  then  minimal  cut  sets  can  be 
determined  by  assuring  that  no  cut  set  is  used  containing  a  smaller  cut  set. 
The  chart  process  further  illustrates  the  number  of  failure  points  within  the 
logic  tree.  This  does  not  appear  too  important  when  using  a  simplistic  logic 
tree  as  the  one  given,  but  for  more  complex  problems,  such  information  takes 
on  new  meaning. 


Table  6 


Boolean  Algebra  Postulates 


1. 

(A  +  0)  *  A 

2. 

A(1 )  =  A 

3. 

AA  =  A 

4. 

A  +  A  =  A 

5. 

A  +  1  =  1 

6. 

A  ( 0 )  =  0 

7. 

*  =  A 

8. 

{A  +  A)  =  1 

9. 

A(A)  =  0 

10. 

A  +  B  =  B  +  A 

11. 

AB  =  BA 

12. 

A  +  {B  +  C)  =  (A  +  B)  +  C 

13. 

A(B  +  C)  =  AB  +  AC 

14. 

A  +  AB  =  A 

15. 

A(A  +  B)  =  A 

16. 

M  =  (J  +  B) 

17. 

I  F  =  (A  +  B) 

18. 

(A  +  IQ)  =  A  +  B 

19. 

A{ft  +  B)  =  AB 

20. 

AB  +  AB"  =  A 

21. 

(A  +  B) (A  +  C)  =  A  +  BC 

22. 

(A  +  C) (A  +  C)  =  A 

T.T 

;.> 

*  a 


The  fact  that  event  A  is  a  single  point  failure  makes  it  most  important 
since  its  failure  alone  will  cause  the  top  event.  However,  if  for  instance 
X-|  or  Y]  were  highly  probable  given  the  probability  levels  of  their  branch 
events,  event  A  may  not  be  the  most  important  point  to  try  to  change  the 
system.  Event  0  occurs  in  two  places,  that  is  in  both  of  the  two  point 
minimal  cut  sets.  It  is  obvious  that  by  changing  the  probability  of  event  D 
one  would  have  a  greater  impact  on  the  whole  system  than  by  changing  events  B 
or  F  even  though  event  B  also  occurs  twice.  Event  B  only  occurs  in  one 
minimal  cut  set. 

It  is  important  to  stress  that  when  using  Boolean  Algebra,  the  alpha 
characters  represent  events  and  that  the  manipulation  of  these  events  is  a 
means  of  determining  those  that  tend  to  be  most  important.  The  analyst  can 
then  go  back  from  the  alpha  characters  to  the  events  themselves  to  see  what 
the  results  are  implying  and  to  determine  whether  or  not  further  reduction  in 
the  probability  of  failure  can  be  undertaken. 

Techniques  for  Hazard  Control 

The  title  of  this  section  is  misleading  as  techniques  will  not  be 
addressed  as  such,  rather,  the  philosophy  of  hazard  control  will  be 
discussed.  Once  a  hazard  has  been  identified  and  analyzed,  something  should 
be  done  about  it.  The  techniques  described  in  the  analysis  section  regarding 
the  identification  of  critical  failure  points  apply  equally  here.  The  most 
critical  failures  need  to  be  addressed  first  in  a  progression  to  the  least 
critical  (see  Figure  4). 

The  most  obvious  method  of  controlling  a  hazard  is  to  eliminate  it 
entirely.  This  is  generally  a  job  for  a  design  engineer  and  may  actually 
entail  a  major  restructuring  or  modification  of  the  system.  This,  in  turn, 
has  a  certain  cost,  not  only  in  terms  of  dollars,  but  also  in  terms  of  time, 
i.e.,  readiness.  If  the  hazard  cannot  be  eliminated  by  a  design  change  for 
some  reason,  then  one  must  seek  some  sort  of  reduction  in  the  hazard.  This 
can  be  accomplished  by  design  changes  that  either  reduce  the  severity  should 
the  hazard  occur,  reduce  the  probability  of  its  occurrence,  or  by  a 
combination  of  the  two. 

Design  changes  are  not  always  possible  or  desirable.  In  some  instances, 
design  changes  may  actually  reduce  the  effectiveness  of  the  system  to  a  point 
where  the  system  would  not  perform  Its  assigned  mission.  Therefore,  in  lieu 
of  design  changes,  hazards  may  be  reduced  by  using  either  safety  or  warning 
devices.  Safety  devices  are  such  things  as  interlocks  which  Inhibit  execution 
In  the  wrong  sequence,  whereas  warning  devices  may  be  a  sound  or  a  flashing 
light  that  signals  an  unsafe  condition. 

If  the  preceedlng  steps  cannot  be  taken,  or  are  taken  but  fail  to 
eliminate  or  control  the  hazard  enough,  the  last  fall-back  position  is  to 
alter  the  procedures  that  effect  the  hazard.  Procedures  may  involve  special 
protective  equipment,  special  training  or  proficiency  trainings,  and  cautions 
and/or  warnings  in  technical  publications. 
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The  order  of  the  steps  given  is  also  the  order  of  their  precedence.  Each 
time  it  is  necessary  to  step  down  from  design  to  safety  devices  to  warning 
devices  to  procedures,  less  control  of  the  hazard  is  exercised.  The  decision 
to  accept  the  hazard  at  a  particular  level  is  generally  left  up  to  the 
manufacturer  or,  in  the  case  of  the  military,  the  program  manager.  After  all 
attempts  to  eliminate  or  control  the  hazard  have  been  applied,  the  final 
acceptance  of  risk  is  based  on  the  need  for  the  system  and  the  mission  that 
the  system  is  to  perform.  Some  systems  such  as  rockets  or  other  types  of 
explosives  are  inherently  hazardous  but  must  be  used  if  a  strong  defense  is  to 
be  maintained. 

Another  point  to  consider  is  that  there  is  seldom  a  completely  safe 
system.  Figure  9  illustrates  this,  showing  that  absolute  safety  is  not 
possible  because  the  cost  of  the  countermeasures  becomes  infinite  in  theory. 
Not  mentioned  in  the  graph  is  the  effect  of  the  countermeasure  on  the 
performance  or  mission  of  the  given  system.  The  object  of  any  manager  is  to 
find  the  point  where  optimal  safety  is  reached  because  that  maximizes  the 
amount  of  safety  available  with  minimum  cost. 

The  cost  of  safety  has  been  discussed  several  times.  How  does  one 
calculate  the  cost  of  safety?  Safety,  after  all,  is  not  something  that 
happens  now,  rather  it  is  something  that  lies  in  the  future.  Not  only  is 
safety  a  future  event,  but  if  it  works  nothing  happens.  Considering  the 
different  steps  that  can  be  taken  to  improve  safety,  i.e.,  reduce  or  eliminate 
a  hazard,  how  can  the  most  cost-effective  technique  be  determined?  Since 
safety  is  a  future  event  that  we  want  to  invest  in,  the  amount  of  time  that 
the  investment  will  work  for  us  is  important.  The  basic  idea  is  that  if  we 
invest  X  amount  of  dollars  now  to  incorporate  a  specific  change  in  a  system 
that  has  a  life  expectancy  of  N  years,  the  overall  savings  in  terms  of  dollars 
not  lost  due  to  accidents/mishaps  will  be  worth  the  initial  X  amount  of 
dollars.  To  figure  out  this  problem  we  can  turn  to  the  economists  for  some  of 
their  computations.  The  series  Present  Worth  Factor  (PWF)  computes  a  future 
value  of  a  series  of  investments  over  time  and  is  given  by: 

(PW  -  i  -  n)  =  H  +  i)n_-  1 

1(1  +  i)n 

Where  i  =  opportunity  cost 
and  n  =  number  of  periods 

The  opportunity  cost  is  actually  the  same  as  the  interest  except  we  are 
redefining  It  to  mean  that  it  is  the  cost  Incurred  should  you  fail  to  select 
the  Improved  safety  proposal. 

For  example,  consider  Figure  10.  Here,  a  system  with  a  life  expectancy  of 
eight  years  is  depicted.  At  time  0  an  Investment  In  dollars  is  made  which  has 
an  effect  in  two  years  and  reduces  the  overall  costs  per  year  for  six  years 
from  level  b  to  level  a.  Was  the  initial  investment  worth  it? 

An  example  with  numbers  will  help  Illustrate  the  utility  of  this 
technique.  Assume  an  engineering  change  proposal  (ECP)  that  costs  $80,000  to 
Install  in  a  system  and  has  a  presumed  life  expectancy  of  seven  years.  Based 
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on  prior  experience,  our  engineers  estimate  that  without  the  change  our  annual 
loss  would  be  about  70K  and  with  the  ECP,  the  loss  would  be  about  50K  per 
year.  This  represents  a  savings  of  20K  per  year  for  seven  years  providing  we 
Invest  80K  now.  Management  estimates  that  inflation  (opportunity  cost)  will 
be  about  10  percent  during  this  period.  So,  what  would  be  the  value  of  140K 
(20K/yr  savings  times  seven  years)  deflated  annually  by  10  percent  for  seven 
years?  The  present  worth  factor  over  this  period  is: 

(PW  -  .10  -  7)  =  H  .+  •))  7  -  1  7  =  4.868 

.1(1  +  .1) 

So  20K  (4.868)  equals  $97,360  and  $97,360  is  greater  than  $80,000  so  we 
would  save  $17,360  by  making  the  change.  In  terms  of  dollars,  the  80K 
investment  is  worth  it. 

However,  if  management  saw  their  opportunity  cost  as  17  percent  rather 
than  10  percent,  the  present  worth  factor  would  then  be: 

(PW  -  .17  -  7)  =  (1  *  ;17)  7  V1  7  =  3.92 

.17(1  +  .17) 

So  20K  (3.92)  equals  $79,449  which  is  less  than  the  80K  investment  so  it 
would  not  be  profitable.  The  obvious  flaw  in  these  calculations  is  the 
accuracy  of  the  estimates,  not  only  of  the  future  cost,  but  also  what  the 
opportunity  cost  might  be.  Future  costs  may  very  well  skyrocket  with  even  one 
law  suit  or  unexpected  recession's  affect  on  Inflation.  The  point  is  that 
given  the  best  Information  available  at  the  time,  an  assessment  can  be  made  as 
to  whether  or  not  a  proposed  expenditure  is  cost-effective.  A  little 
imagination  will  allow  you  to  see  other  possible  uses  for  this  cost/benefit 
type  of  analysis.  You  could  perform  these  computations  for  all  variations  of 
proposed  means  for  hazard  elimination  and/or  reduction  to  determine  which  one 
is  the  most  cost-effective. 

Once  the  most  cost-effective  proposal  has  been  identified,  the  manager 
must  decide  If  that  Is  the  proposal  that  should  be  implemented.  One  more 
consideration  that  must  be  taken  Into  account  before  such  a  decision  is  made, 
is  the  social  cost  involved.  There  Is  no  way  to  measure  this  in  terms  of 
dollars  and  cents.  It  consists  of  things  such  as  public  good  will  or  company 
reputation.  The  most  cost-effective  procedure  may  also  produce  the  most 
environmental  pollution  or  be  based  on  the  fewest  number  of  law  suits  due  to 
consumer  Injury  or  death.  Such  social  concerns  may  lead  a  manager  to  actually 
Implement  a  more  costly  procedure  simply  to  reduce  the  public's  possible 
disfavor. 

Maintenance  and  the  Man 

Systems  that  require  maintenance  require  malntainers  who  are  able  to 
perform  the  required  tasks.  It  is  very  difficult  to  describe  the  "average" 
maintalner.  If  we  design  a  maintenance  task  for  the  50th  percentile  person, 
does  this  mean  that  the  3rd  and  98th  percentile  person  can  also  perform  the 
task?  What  about  women?  Obviously,  the  50th  percentile  (average)  man  is  not 
the  same  as  the  50th  percentile  woman.  In  fact,  the  50th  percentile  person 
does  not  exist,  rather  the  phrase  is  used  to  represent  the  mean  figures  on  a 
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number  of  variables  such  as  height,  weight,  reach,  strength,  education,  IQ, 
shoe  size,  etc.  Any  attribute  you  can  conceive  has  a  mean  value  which 
represents  the  "average"  person.  The  fact  that  we  have  a  selected  population 
within  the  naval  maintenance  force  confuses  the  problem  somewhat  since  we  have 
to  know  what  the  extreme  values  of  our  population  are  in  order  to  define  the 
"average"  person. 

The  problem  of  designing  a  maintenance  task  for  the  "average"  person  is 
something  that  has  yet  to  be  mastered.  As  more  and  more  information  about  the 
maintainer  population  Is  defined,  the  task  becomes  more  realistic.  It  is 
doubtful  if  all  the  information  will  ever  be  known.  There  will  always  be  some 
problems  with  exceptional  situations  that  occur  so  infrequently  as  to  not 
warrant  any  large  expenditure  of  funds  in  attempting  to  identify  them. 

The  fact  that  the  "average"  man  is  so  elusive  is  only  one  of  the 
problems.  Another  major  consideration  is  the  fact  that  human  beings  as  a 
species  have  certain  limits  in  their  abilities.  These  include  information 
processing,  sensation,  perception,  and  strength.  One  of  the  major  tasks  of 
design  engineers  is  to  develop  tools  which  extend  these  limits  and  help  to 
include  the  majority  of  the  maintainer  population.  In  order  to  do  this,  the 
engineers  must  know  what  these  limits  are  so  that  their  designs  not  only  can 
extend  them  when  necessary,  but  also  so  that  the  tools  reach  the  minimum  or 
maximum  limits  to  begin  with.  After  all,  what  good  is  a  pair  of  pliers  if  a 
5th  percentile  woman  cannot  squeeze  them  enough  to  perform  the  required 
function?  It  may  be  a  matter  of  leverage  or  surface  area  that  needs  to  be 
considered.  The  same  applies  to  Information  processing.  If  a  task  is  too 
hard  to  learn  for  a  group  of  maintainers,  perhaps  either  redesigning  the  task 
or  rewriting  the  instructions  on  how  to  perform  it  would  be  beneficial. 

Insuring  that  a  maintainer  can  perform  his  required  tasks  as  easily  as  is 
functionally  possible  is  a  large  step  in  attempting  to  provide  a  safe  working 
environment.  When  maintainers  are  performing  their  tasks,  they  are  generally 
devoting  most  of  their  attention  towards  the  proper  execution  and  completion 
of  the  task  and  so  expose  themselves  to  hazards  which  would  normally  be 
avoided  without  much  thought.  The  obvious  example  would  be  the 
troubleshooters  trying  to  perform  their  duties  on  a  flight  deck  during  a  night 
launch  In  marginal  weather.  It  takes  little  imagination  to  see  that  the 
maintainer  might  be  injured  due  to  jet  blasts  or  spinning  propellers.  The 
more  their  job  has  been  designed  to  be  easy,  the  more  attention  they  will  be 
able  to  devote  to  their  surroundings.  Thus,  it  is  imperative  that  the  design 
engineer  design  his  system's  maintenance  functions  to  a  worst  case  situation 
for  the  specified  population  of  maintainers  servicing  that  system.  Anything 
less  is  failing  to  maximize  safety  and  enhances  the  loss  control  for  the  total 
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An  overview  of  our  Design  for  Effective  Maintenance  study  (DEM)  is 
presented  in  Table  1.  The  DEM  study  is  a  part  of  the  NADC  Design-for- 
Maintainers  program.  DEM  attempted  to  do  two  things  in  particular.  First,  it 
was  a  feasibility  study  to  determine  whether  useful  data  exists  in  the  Naval 
Safety  Center's  data  base.  "Useful"  is  defined  here  as  the  extent  to  which 
data  can  result  in  what  we  term  Effective  Maintenance  Design  Features  (EMDF), 
design  principles  concerning  the  human  factors  of  maintainability. 

Second,  in  a  broader  sense,  we  wished  to  develop  a  methodology  applicable 
to  Naval  Safety  Center  and  other  data  bases,  a  "model,"  which  when  applied  to 
data  will  point  to  EMDFs.  In  the  remainder  of  our  presentation  here,  we  wish 
to  show  that  both  of  these  objectives  were  met.  That  is,  the  Naval  Safety 
Center  data  base  certainly  contains  useful  data,  and  we  were  able  to  develop  a 
valid  model  of  how  to  approach  such  data. 

Our  overview  of  the  Naval  Safety  Center  is  presented  in  Table  2.  Its 
stated  mission  is  to  preserve  resources  (both  material  and  personnel)  through 
the  functions  of  detecting  hazards,  and  then  eliminating  them  through  analysis 
and  monitoring  of  corrective  action.  Analysis,  of  course,  is  the  key,  and  so 
this  function  is  implemented  through  the  Accident  and  Mishap  Data  Base  (AMD). 
The  AMD  provides  for  a  very  large  amount  of  data  to  be  recorded  about  each 
Naval  Air  mishap;  these  are  coded  by  Safety  Center  personnel  operating  under 
OPMAVIHST  3750.6M. 

As  we  reviewed  the  data  format  and  variable  definitions  of  the  AMD,  we 
discovered  26  classes  of  variables  relevant  or  potentially  relevant  to 
maintenance.  By  "variable  classes,"  we  mean  to  indicate  that  many  variables 
have  several  versions,  such  as  "First  Involved  Component,"  "Second  Involved 
Component,"  etc.  In  all,  there  were  66  variables  identified  as  relevant. 

We  also  requested  and  obtained  mishap  narrative  data  up  to  3000  characters 
long.  All  mishap  records  have  a  mishap  narrative,  but  most  are  less  than  200 


words  long.  The  narratives  verbally  describe  the  circumstances  of  the 
mishaps,  and  thus  provide  an  invaluable  opportunity  to  attempt  to  predict 
"situations"  (narrative  reports  describing  common  maintenance  problems)  from 
objective  data. 

Finally,  we  found  that  the  Naval  Safety  Center  has  indexed  a  subset  of  the 
AMD  according  to  certain  maintenance  indicators  and  personnel  causal  factors 
where  all  members  of  the  subfile  have  some  maintenance  component  to  the 
mishap.  This  file  is  termed  the  Maintenance  Malpractice  File,  and  we 
requested  these  records  specifically  rather  than  the  entire  AMD. 

Table  1 

Design  for  Effective  Maintenance  (DEM) _ 

•  Part  of  overall  Design-for-Maintainers  program 

•  Objectives: 

•  Evaluate  feasibility  of  using  existing  Naval  Safety 
Center  (NSC)  data  to  extract  Effective  Maintenance  Design 
Features  (EMDFs) 

•  Develop  a  methodology  for  extracting  EMDFs  from  the  NSC 
data  base  and  other  data  bases 


total  of  5886  cases.  We  then  looked  at  the  univariate  frequency  distributions 
in  a  search  for  directions  for  further  analysis.  We  had  originally  hoped  to 
recode  the  data  in  such  a  way  that  multiple  regression  and  its  family  of 
related  techniques  (factor,  cluster  and  discriminant  analysis)  could  be 
applied  to  the  set.  However,  we  found  in  looking  at  the  distributions  that 
the  degree  of  missing  data  (i.e.,  virtually  no  case  record  had  data  entered 
for  each  variable)  and  the  nominal-level  nature  of  the  data  preclude  this. 

Our  later  analyses,  then,  attempted  to  determine  essentially  the  same  things, 
associativity  among  variables,  through  nonparametric  means,  especially 
cross-tabulation. 

In  cross-tabulating,  we  set  our  goal  as  one  of  determining  clusters  of 
related  systems  and  maintenance  factors.  This  is  a  simple  point,  equivalent 
to  saying  that  we  would  like  to  know  what  hardware  (e.g.,  a  specific  component 
on  a  given  aircraft)  is  associated  with  what  maintenance/personnel  "causes"  of 
mishaps.  This  assumes  that  such  incidents  happening  consistently  contain 
useful  information  about  potential  human  factors  design  flaws  in  terms  of 
maintainabil ity. 

We  then  wished  to  develop  a  statistical  model  of  the  data  out  of  which 
would  fall  cases  with  this  presumed  useful  information.  By  "model,"  we 
originally  conceived  of  regression  models  for  selection;  with  the 
nonparametric  schemes  we  adopted,  "model"  refers  to  the  techniques  we  used  to 
define  a  related  cluster  of  cases.  We  then  reviewed  those  cases'  narratives 
("folded  back"  the  objective  data  on  the  subjective  data)  and  attempted  to 
derive  FMDF  hypotheses.  From  this  point,  we  can  outline  ways  to  test  those 
hypotheses  which  will  result  in  true  EMDFs. 

Table  3 

Study  Approach 


•  Obtain  Maintenance  Malpractice  File  for  five  years 
(1977-1981) 

•  Frequency  distributions  of  each  variable  for  "clues," 
directions,  hypotheses 

•  Cross- tabular  analysis  within  and  between  variable 
classes  for  associativity  of  Systems  and  Maintenance 
factors 

•  Model  for  selection  of  cases  for  qualitative  review 

•  EMDF  hypotheses  based  on  "foldback" 

•  Di reckons;  future  work 

Table  4  and  Figure  1  describe  and  show  how  our  modeling  approach 
proceeded.  Beginning  with  a  relatively  amorphous  data  base  (requiring 
substantial  exploration  to  understand  its  structure  and  content),  we  sorted 
variables  into  those  containing  systems  and  maintenance  information.  We  then 


cross-tabulated  these  variables,  and  adopted  numerous  (Iterative)  criteria 
toward  defining  what  a  "large"  cell  was.  When  a  large  cell  was  found,  we 
broke  out  the  case  narratives  pertaining  to  that  system-by-maintenance 
combination.  Examining  these  and  applying  to  them  a  process  of  subjective 
Integration  results  in  an  EMDF  hypothesis. 

It  Is  at  this  point  that  the  scope  of  this  present  study  ends.  However, 
we  feel  that  a  microscopic  examination  of  the  hardware  and  maintenance 
environment  bearing  on  the  hypothesis  can  lead  to  validation  of  the  EMDF.  The 
steps  of  such  a  follow-through  examination  are  described  later  and  in  Table  9. 

Table  4 

General  Approach  to  Modeling 

Elements 

•  Associative  techniques 

•  Data  reduction 

•  Foldback  analysis 

•  Judgments/recommendations 

Figure  2  depicts  the  specific  data  we  focused  on  in  our  model 
development.  The  variables  of  interest  are  termed  "Material  Special  Data" 
(MSD) ,  and  they  code  "types  of  occurrences  which  are  encountered  in  mishap 
analysis."  There  are  five  such  variables  (first,  second  ...  fifth  MSD),  and 
we  chose  to  use  the  first,  second  and  third  due  to  large  amounts  of  missing 
data  in  the  fourth  and  fifth. 

Each  MSD  variable  has  two  types  of  codes.  One  type  consists  of  three 
alpha  characters  (e.g.,  "AAA";  BBP"),  which  refers  to  broad  comments  about  a 
mishap  of  a  maintenance  nature.  For  example,  "AAA"  means  "Maintenance  error  - 
general"  and  "BBP"  means  "Improper  use  of  a  safety  or  locking  device  (with 
photographs)."  The  other  type  consists  of  an  alpha  and  two  numeric  characters 
(e.g.,  "L10"),  referring  to  a  specific  subsystem.  For  example,  "L10"  means 
"landing  gear  over  torque."  The  MSD  variables,  then,  contain  both  the  systems 
and  maintenance  information  called  for  In  our  approach. 

We  pulled  these  data  apart  by  writing  a  program  to  make  a  pass  through  the 
data  and  look  for  the  alpha  and  alphanumeric  codes.  When  it  found  an  alpha 
code,  the  program  placed  it  in  a  new,  all -alpha  variable  and  deleted  it  from 
the  root  variable.  When  It  encountered  an  alphanumeric  code,  it  left  it  alone 
and  left  blank  that  position  in  the  all-alpha  variable.  The  result  was  two 
variables,  the  root  one  containing  only  alphanumberlc  codes  (system 
Information)  termed  SI  and  the  second  containing  only  alpha  codes  (maintenance 
information)  termed  Ml.  Then  the  process  was  repeated  for  the  second  and 
third  MSD  variables,  resulting  in  S2,  M2,  S3  and  M3. 

The  table  at  the  bottom  of  Figure  2  shows  how  these  six  variables  were 
cross- tabulated,  with  each  systems  variable  crossed  with  each  maintenance 
variable.  Note  that  the  SI  x  Ml ,  S2  x  M2  and  S3  x  M3  tables  make  no  sense. 
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Figure  1.  Model  development  process 
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since  each  of  these  pairs  is  complementary  within  the  pair  or  "zero 
correlated"  in  a  nonparametric  sense.  That  is,  a  case  with  an  entry  in  SI  by 
definition  has  a  blank  in  Ml;  if  it  has  a  code  in  M2  it  must  have  a  blank  in 
S2,  etc.  The  result  is  six  useful  tables,  arbitrarily  termed  1-6  in  Figure  2. 

In  Table  5  we  describe  the  results  of  our  analysis  of  the  cross-tabulation 
tables  1-6.  We  selected  system  codes  which  met  a  number  of  rather  stringent 
criteria,  including  size  (number  of  cases  in  the  cell),  at  least  50  percent  of 
such  system  codes  co-occuring  with  the  alpha,  maintenance  codes,  small  amounts 
(<50  percent)  of  missing  data  and,  in  some  cases,  further  component  code  data 
to  narrow  down  the  system  being  described.  The  effect  of  these  criteria  was 
to  reduce  the  number  of  system  codes  by  a  factor  of  12.5.  Then  we  ranked  the 
codes  meeting  all  of  these  criteria  according  to  the  number  of  tables  in  which 
they  did  so;  the  systems  eventually  examined  met  the  criteria  in  3,  3,  and  2 
tables,  respectively.  Finally,  when  we  selected  three  multitable  codes  which 
survived  all  of  the  criteria,  v/e  examined  the  distributions  of  those  code 
combinations  by  aircraft,  and  listed  the  case  record  codes  of  those 
systems-by-maintenance  combinations  which  co-occurred  with  the  most  frequent 
aircraft. 

The  case  record  codes  were  used  to  access  the  actual  case  narratives  of 
the  mishaps  (which  were  delivered  to  us  on  hard  copy  for  convenience  of 
access).  The  three  codes  focused  upon  involved  an  ejection  seat,  external 
tanks,  and  an  airspeed/altimeter  pitot  line.  These  narratives,  and  the  EMDF 
inferences  we  drew  from  them,  are  summarized  in  Tables  6,  7  and  8. 

Table  5 

_ Results _ 

•  Three  subsystem  codes  selected  on  basis  of: 

•  Humber  of  cases 

•  Maintenance  personnel  involvement  codes 

•  Sufficiently  low  level  of  missing  data 

a  Further  component  data 

•  Case  narratives  reviewed 

I  Ejection  seat  (inadvertent  actuation) 

•  Pin  security 

•  Resemblance  to  brake 

II  External  tanks  (inadvertent  ground  jettison) 

•  Pin  security 

III  Airspeed/altimeter  pitot  line  (instrument  failure) 

•  Instrument  panel  access 

•  Drain  plug 

a  100%  "hit  rate"  on  applications  of  model  thus  far 


ft: 


Table  6  abstracts  the  case  narratives  relating  to  the  escape  seat 
situation  uncovered.  From  them,  two  common  themes  emerge  with  potential 
bearing  on  EMDFs.  The  first  is  that  even  experienced,  cockpit-qualified 
personnel,  very  familiar  with  the  systems  (e.g.,  "brakemen")  sometimes  mistake 
the  seat  firing  handle  for  the  emergency  brake  handle  with  extremely  dangerous 
consequences.  The  second,  a  prerequisite  for  any  of  these  mishaps,  is  that 
the  safety  pins  are  not  always  fully  in  position.  When  they  are  not  (and  they 
are  almost  always  supposed  to  be  when  the  aircraft  is  on  the  ground),  a 
variety  of  handle-pulling,  linkage-bumping,  etc.,  errors  can  cause  inadvertent 
actuation. 

Pealing  with  the  pin  security  issue  first,  it  would  appear  that  two  areas 
for  further  study  are  obvious.  The  first  is  that  sensors  could  be  built  into 
the  pin  holes  such  that  when  (a)  the  plane  is  on  the  ground  and  (b)  the  seat 
is  not  fully  pinned,  a  cockpit  alarm  (visual,  auditory  or  both)  is  set  off, 
for  the  duration  of  the  dangerous  state  (until  the  pins  are  replaced).  The 
second  possibility  is  to  redesign  pins  so  that  they  may  be  removed  only  by  use 
of  a  special  tool  or  key,  and  then  restrict  access  to  the  key.  Were  only 
maintenance  supervisors  able  to  "arm"  the  seat,  errors  of  lack  of  experience 
and/or  training  would  greatly  diminish  or  disappear. 

As  far  as  the  firing  handle's  resemblance  to  the  brake  is  concerned,  this 
is  a  question  requiring  field  inspection  to  evaluate.  Whatever  the 
resemblance  in  position,  color,  size,  "feel,"  tension,  etc.,  design 
modifications  should  be  feasible  to  reduce  these  errors  and  mishaps. 

Table  6 

Case  I 


•  D31/D02  (ejection  seat) 

•  Ten  cases 

•  All  on  ground 

t  "No  pin  in;  fatal;  procedure  violation" 
t  "Asked  unqualified  person  to  pull  brake" 

•  "Pulled  wrong  handle  during  hydraulic  checks" 
e  "Night  brake  technician  with  no  flash  light" 

•  "Oxygen  bottle  ignited;  blew  seat" 

•  "Brake  rider  mistook  handle" 

•  "Personnel  bumped  actuator  (not  pinned)" 
t  "Jacket  snagged  on  cables/linkage" 

•  "Using  handle  as  handhold  (not  pinned)" 

•  "Rider  told  to  release  brakes  (not  pinned)" 


•  Cockpit  status  alarm  (pin  sensors) 

•  Locking  pins  (tool;  key) 

•  Handle  redesign 


Table  7  contains  abstracts  of  case  narratives  relative  to  inadvertent 
actuation  of  an  auxiliary  tank  release  system.  All  seven  mishaps  occurred  on 
the  ground  during  a  regular  morning  inspection  of  the  system.  It  appears  that 
the  test  procedure  for  this  aircraft  calls  for  de-arming  the  tanks  by  checking 
the  breeches  and  pulling  the  leads  which  go  to  the  explosive  charges.  Then, 
the  leads  are  connected  to  a  test  set,  the  system  activated  and  the  system 
operation  and  continuity  check  performed. 

A  common  error  seems  to  be  for  one  or  more  of  the  charges  to  be  de-armed 
but  not  the  entire  set  of  five.  Both  novice  and  experienced  maintenance 
personnel  are  subject  to  this,  although  "experience,"  "training," 
"supervision,"  and  "procedure  violation"  are  usually  cited.  The  consequence 
is  usually  for  one  side  of  a  fuel  tank  to  drop,  pivoting  the  other  end  against 
the  pylons,  striking  the  floor  with  ensuing  tank  rupture  and  fuel  spillage. 

As  with  the  escape  seat  activation  example  (Case  I),  the  problem  seems  to 
have  to  do  with  safety  pin  security  and,  in  general,  status  information  about 
the  equipment  involved  in,  and  thus  the  consequences  of,  the  test.  Status 
alarms  could  be  built  into  the  cockpit  and  provide  "continue/stop"  displays 
when  an  early  event  in  the  test  sequence  occurs.  And,  the  test  sets 
themselves  could  provide  alarms  when  the  test  is  about  to  occur  when  the 
aircraft  is  not  fully  de-armed.  And  finally,  locking  pins  releasable  only  by 
tool  or  key  held  by  maintenance  supervisors  would  encourage  that  greater  care 
and  expertise  be  brought  to  bear  on  these  tests. 

Table  7 

Case  II 


•  G32  (bomb  rack  release  system;  droppable  fuel  tanks) 

•  Seven  cases 

•  All  on  daily  inspection 

•  "Failed  to  insure  rack  pinned" 

•  "Two  of  five  not  de-armed" 
t  "One  of  five  not  de-armed" 

•  "One  of  five  not  de-armed" 

•  "Breeches  not  checked" 

•  "Returned  to  wrong  plane" 

•  "Failed  to  de-arm" 

•  Central  alarm  to  pin/de-arming  mechanism 
(cockpit;  test  equipment) 

•  Locking  pins  (tool;  key) 


Our  third  case  study  is  presented  in  Table  8.  Here,  sixteen  narratives 
v/ere  found  pertaining  to  a  single  code  (False/Erratic  Instrument  Indication) 
and  nine  of  these  dealt  specifically  with  the  static  pitot  tube  system.  (A 
pitot  tube  Is  a  device  for  transferring  pressure  changes  and  is  used  in  the 
barometric  altimeter  system  and  the  airspeed  Indicator.) 


When  we  reviewed  the  nine  pitot  tube  case  narratives,  we  found  that  they 
fall  neatly  into  two  clusters,  six  dealing  with  crimped,  pinched  and  loose 
lines  where  the  tube  mates  with  the  altimeter,  and  three  dealing  with  water  in 
the  pitot  tube  itself.  Table  8  is  organized  according  to  these  clusters. 

The  first  cluster,  crimped  lines,  is  an  example  of  what  is  termed  "type  d" 
maintenance  error,  that  is,  the  damaging  of  equipment  in  the  process  of 
repair.  The  narratives  indicate  that  the  cramped  design  of  equipment 
arrangement  behind  the  panel  may  contribute  to  these  errors.  It  appears  that 
as  instruments  are  replaced,  and  specifically  as  the  pitot  tube  line  is  shoved 
back  through  the  panel,  the  line  is  damaged. 

Clearly,  space  is  at  a  premium  in  a  modern  aircraft,  but  at  some  point  an 
alternative  design  which  minimizes  the  opportunity  for  pinching  pitot  lines 
should  be  considered.  Specifically,  one  would  think  that  a  tubing  guide  or 
tensioned  retracting  mechanism  could  be  utilized  to  guide  the  line  past  other 
rear-of-panel  equipment.  The  second  cluster,  v/ater  in  the  pitot  tube,  is 
associated  with  the  low  point  drain  plug  on  the  outside  pitot  line. 

Apparently,  water  collects  and  condenses  inside  the  tube,  and  a  routine 
maintenance  action  is  to  remove  a  plug,  drain  the  water,  and  replace  the  plug 
cap;  failure  to  replace  the  cap  causes  the  mishaps.  A  direction  to  pursue 
toward  an  EMDF  is  for  more  fail-safe  mechanisms,  either  a  chain  making  the  cap 
captive  or  some  sort  of  spring-loaded  plunger  mechanism  which,  after  being 
actuated  to  drain  the  water,  would  automatically  move  back  into  place. 

Table  8 
Case  III 


•  134  {false/erratic  instrument  indication) 

•  9  of  the  16  cases  involved  the  pitot  tube 

•  "Pitot  line  crimped  at  altimeter  connection" 

•  "Pinched  static  line  behind  altimeter" 
t  "Loose  pitot  fittings" 

•  "Kink  in  altimeter/airspeed  static  line" 

•  "Incorrect  mounting  screws  pierced  pitot  static  probe" 

•  "Static  hose  twisted  and  crimped" 

•  Pitot  connections  on  back  of  airspeed  indicator  and 
altimeter 

•  Pitot  connection  change;  retracting  mechanism  -  tubing  guide 

•  "Pitot  drain  cap  removed" 

•  "Low  point  drain  plug  missing" 

•  "Water  in  pitot  system" 


•  Failsafe  pitot  drain  mechanism 


From  this  point  of  analysis,  where  do  we  proceed  to  validate  these 
hypotheses  and  generate  true  EMDFs?  Table  9  lists  the  steps  we  see  at  this 
time.  First,  we  can  attempt  to  find  out  more  about  these  specific  cases  and 
all  other  cases  of  the  same  type  (which  may  not  have  fallen  out  of  our  model 
readily).  Then,  we  feel  it  is  important  to  review  (where  possible) 
information  gathered  from  the  manufacturer  and  design  team,  to  understand  (a) 
why  the  design  was  implemented,  (b)  whether  we  fully  understand  the  situation 
and  (c)  whether  potential  design  changes  are  "don't  cares"  or  if  in  fact,  they 
might  adversely  affect  other  systems. 


In  field  evaluation,  we  see  both  inspection  (observation  of  maintenance; 
hands-on  experience  for  human  factors  engineers)  and  interviews  with 
maintainers  to  be  useful  activities.  Finally,  the  scope  and  potentially  a 
cost-benefit  understanding  of  the  problems  and  their  solution  can  be  derived 
from  other  data  bases,  especially  3-M  which  documents  all  Naval  Air 
maintenance  (not  just  maintenance  causing  mishaps).  From  all  of  these 
activities  we  hope  that  general  design  standards  (EMDFs)  will  be  able  to  be 
validly  explicated. 

Table  9 

_ EMDF  Derivation _ 

•  Review  of  detailed  case  records 

•  Manufacturer  (design)  review 

•  Field  inspection 

•  Field  interviews 

•  Validation  with  other  data  bases  (3-M) 

•  Design  standards 


To  conclude  (Table  10),  we  feel  that  our  Design  for  Effective  Maintenance 
study  has  produced  three  reasonable  examples  of  potential  maintainability 
design  flaws  (EMDF  hypotheses)  drawn  from  existing  data  (AMD).  Since  the 
model  was  only  applied  three  times,  we  have  every  reason  to  believe  our 
approach  would  be  successful  in  producing  numerous  (perhaps  hundreds  of) 
others.  The  methodology  is  not  restricted  to  the  Material  Special  Data 
approach  taken  herein,  and  other  variable  combinations  within  the  AMD  might  be 
even  more  successful.  And,  there  is  no  apparent  reason  why  our  approach  could 
not  be  applied  to  other  data  bases,  especially  3-M.  We  feel  that  this  line  of 
work,  followed  through  in  the  manner  described  earlier,  can  have  large  and 
direct  benefit  to  the  overall  Design-for-Maintainers  program  (and  the  human 
factors  of  maintainability  in  general)  and  we  are  currently  working  on 
specific  proposals  along  those  lines. 
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Table  10 
Conclusions 


•  EM OF  hypotheses  generated  for  specific  subsystems 

•  Methodology  developed  applicable  to  other  subsystems 
in  MSC/AMD 

•  Approach  developed  applicable  to 

•  Other  variable  combinations  within  NSC/AMD 

•  Other  data  bases  (e.g.,  3-M) 


Acknowledgement 


Harris  Government  Electronic  Systems  Division  is  a  hardware-oriented 
producer  of  a  variety  of  systems  for  government  customers,  and  is  involved  in 
direct  technical  support  of  these  programs.  John  Schmitt  is  an  Associate 
Principal  Engineer  (Human  Factors/Safety).  Larry  Lamb  is  also  an  Associate 
Principal  Engineer,  and  in  addition  has  group  leader  responsibility  for  the 
Human  Factors/ Systems  Safety  group.  John  Mocharnuk  is  an  Engineering 
Psychologist  and  the  head  of  the  Systems  Effectiveness  section,  which  includes 
Human  Factors/Safety,  Rel iabil ity/Maintainabil ity ,  and  EMI/EMC/TEMPEST. 


1 


1* 

%  **• 


„-V;  - 


Design,  Implementation,  and  Evaluation  of 
Approaches  to  Improving  Maintenance  Through  Training 


William  B.  Rouse 


Center  for  Man-Machine  Systems  Research 
Georgia  Institute  of  Technology 
Atlanta,  Georgia 


Maintenance  is  often  discussed  as  a  serious  problem.  In  the  military, 
poor  maintenance  is  cited  as  a  cause  of  low  levels  of  operational  readiness. 

In  the  private  sector,  maintenance  problems  have  been  noted  as  the  cause  of  a 
shift  away  from  American-produced  goods.  The  sources  of  the  maintenance 
problem  include  both  increasingly  complex  equipment  systems  and  decreasing 
skill  levels  of  maintenance  technicians. 

Figure  1  depicts  the  relationships  between  three  types  of  remedial 
measures  and  maintainer  (or  operator)  performance.  Selection  involves  using 
tests  of  ability,  aptitude,  and  perhaps  even  cognitive  style  to  Identify 
potential  trainees  who  are  likely  to  excel,  or  at  least  be  adequate,  as 
maintenance  technicians.  Training  involves  providing  the  trainee  with  facts, 
principles,  and  experiences  that  will  enable  him  to  achieve  maintenance 
performance  objectives.  Aidi ng  denotes  all  aspects  of  the  system  design  which 
are  provided  to  enhance  maintainer  performance,  Including  test  points, 
modularity,  test  equipment,  procedures,  etc. 

Figure  1  shows  several  loops  feeding  back  from  performance  to  selection, 
training,  and  aiding.  Ideally,  these  three  processes  should  be  responsive  to 
performance  objectives.  However,  at  least  In  the  military,  the  responsiveness 
of  selection  is  currently  limited  by  the  population  available  and  the 
responsiveness  of  aiding  is  limited  by  the  lengthy  procurement  process. 
Training,  on  the  other  hand,  should  be  very  responsive  to  performance 
feedback.  This  is  due  to  the  fact  that  the  military,  as  well  as  much  of 
Industry,  designs  and  conducts  its  own  training  programs  and  hence,  should  be 
able  to  adapt  these  programs  to  performance  requirements.  A  mechanism  by 
which  this  adaptation  might  be  achieved  is  discussed  in  the  following  section. 
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Figure  1.  Relationship  of  selection,  training,  and  aiding  to  performance. 


Synthesis  of  Training  Programs 


Figure  2  shows  the  three  phases  of  program  synthesis  and  their 
relationship  to  trainee  performance.  Design  includes  choosing  a  maintenance 
philosophy  regarding  the  role  of  the  maTntaTner,  choosing  performance 
objectives  and  determining  the  skills  necessary  to  achieve  these  objectives, 
and  considering  alternative  training  methods  and  technologies.  Implementation 
involves  sequencing  and  coordinating  objectives  with  respect  to  a  particular 
equipment  system,  integration  of  methods  and  technologies,  and  development  of 
a  performance  measurement  plan.  Evaluation  includes  assessing  trainee 
performance  both  during  training  and  subsequently  on  the  job,  as  well  as 
estimating  the  impact  of  maintainer  performance  on  system  performance. 
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Figure  2.  Synthesis  of  training  programs. 


Figures  1  and  2  represent  two  processes  that  are  structurally  similar. 
Both  employ  feedback  to  compare  desired  and  actual  performance,  and  both  are 
assumed  to  have  some  mechanisms  for  adapting  the  process  so  as  to  produce 
actual  performance  that  Is  close  to  the  desired  performance.  Thus,  the  three 
essential  features  of  the  representations  In  Figures  1  and  2  are:  1) 
definition  of  desired  performance,  2)  feedback  of  actual  performance,  and  3) 
mechanisms  for  adapting  the  process.  These  three  ingredients  must  be  present 
If  the  approach  to  synthesizing  training  programs  shown  in  Figure  2  is  to  be 
successful . 


Unfortunately,  this  approach  must  be  viewed  as  an  idealization  rather  than 
reality.  The  basic  difficulty  is  that  desired  maintainer  performance,  if  it 
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Is  defined  at  all,  Is  defined  In  terns  of  global  measures  such  as  MTTR  (mean- 
£  tlne-to-repalr),  MEOF  (no  evidence  of  failure)  rate,  and  RTOK  (retest  ok) 

rate.  These  measures  are  not  sufficiently  diagnostic  to  provide  for  an 
adaptive  mechanism,  a  reasonably  clear  path  of  adaptation.  A  further 
5?  difficulty  is  that  performance  feedback  Is  often  nonexistent  or  is  so  highly 

S;  aggregated  that  it  can,  at  best,  only  serve  as  a  warning  device  that  a  class 

%  of  equipment  systems  Is  experiencing  maintenance  problems. 

fv  While  one  might  argue  that  the  success  of  the  approach  to  program 

*»  synthesis  depicted  in  Figure  2  is  also  limited  by  a  lack  of  a  knowledge  base 

upon  which  the  design  of  adaptive  mechanisms  can  be  founded,  this  is  really  a 
D  secondary  difficulty.  Until  desired  maintainer  performance  is  defined  at  an 

appropriate  level  and,  until  performance  feedback  of  suitable  measures  is 
instituted,  the  knowledge  base  cannot  be  expanded  and  certainly  not 
, ,  exploited.  The  central  problem  is  defining  and  measuring  performance. 
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■  Maintainer  Performance 
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Discussions  of  performance  usually  focus  on  the  overall  objective  of 
system  performance  expressed  In  terms  of  both  capabilities  and  availability  or 
operational  readiness.  These  global  measures  are  affected  by  five  constituent 
subsystems:  hardware,  software,  support,  operators,  and  maintainers.  Thus, 
the  maintainer  makes  only  one  of  many  contributions  to  system  performance,  and 
overall  measures  of  system  performance  do  not  necessarily  provide  clear 
Insights  into  the  maintainer' s  contribution. 

As  noted  earlier,  overall  maintenance-oriented  performance  measures  such 
as  MTTR,  NEOF,  and  RTOK  also  do  not  provide  great  insights  into  maintainer 
performance.  What  is  necessary  Is  an  understanding  of  the  process  which  the 
maintainer  goes  through  In  achieving  some  particular  level  of  overall 
maintenance  performance.  There  appear  to  be  two  rather  different  ways  of 
gaining  this  understanding:  task  analysis  and  modeling. 

For  frequent  or  well-defined  abnormal  and  emergency  situations,  the 
required  sequence  of  human  observations,  decisions,  and  actions  can  be 
determined  via  task  analysis.  Specific  behavioral  objectives  can  then  be 
chosen  and  appropriate  procedures  designed.  There  are  many  excellent  examples 
of  where  this  task  analysis  approach  to  defining  maintainer  (or  operator) 
performance  has  succeeded  admirably. 

However,  while  task  analysis  may  be  viewed  as  necessary,  it  is  not 
sufficient.  For  Infrequent  or  Ill-defined  situations,  or  for  multi-event 
situations,  the  required  sequence  of  observations,  decisions,  and  actions  may 
be  very  difficult  to  determine.  Indeed,  it  may  even  be  very  difficult  to 
define  the  nature  of  such  situations. 

This  aspect  of  the  maintainer' s  role  is  best  viewed  as  problem  solving 
(Rouse,  1981,  1982).  A  reasonable  approach  to  understanding  the  maintainer  as 
a  problem  solver  is  through  the  use  of  models  that  attempt  to  capture  the 
essence  of  the  problem  solving  skills  required  for  maintenance  tasks.  Such 
models  can  be  used  as  a  basis  for  devising  and  evaluating  the  dimensions  of 
problem  solving  performance  relevant  to  maintenance  tasks.  These  dimensions 
can  provide  Insight  Into  the  problem  solving  principles  (as  opposed  to 
procedures)  that  maintenance  technicians  need  to  know  in  order  to  succeed  in 
Infrequent,  ill -defined,  or  multi -event  situations. 


123 


A  recent  study  of  a  wide  variety  of  measures  of  problem  solving 
performance  In  maintenance  tasks  (Henneraan  &  Rouse,  1982)  concluded  that  there 
are  three  basic  dimensions  of  performance:  time,  error,  and  inefficiency. 

Time  refers  to  the  cumulative  "active"  time  required  for  a  maintenance 
operation.  Errors  are  defined  as  observations,  decisions,  or  actions  that 
result  In  no  progress  relative  to  the  goal  of  the  maintenance  operation. 
Inefficiencies  are  observations,  decisions  or  actions  that  yield  progress,  but 
not  as  much  progress  as  Is  possible.  {For  further  discussion  of  similar 
definitions,  see  also  Duncan  &  Gray  [1975]  and  Hunt  S  Rouse  [1981].) 

Measures  of  performance  on  the  dimensions  of  time,  errors,  and 
inefficiency  should  provide  the  insights  necessary  to  isolate  the  sources  of 
problems  detected  in  terms  of  high  MTTR,  NEOF,  or  RTOK.  Time  Is  relatively 
strai ghtforward  to  measure,  although  “active"  time  can  be  difficult  to 
identify.  Identifying,  and  particularly  classifying,  errors  can  be  a  rather 
Intensive  process,  but  provides  multifaceted  insights.  Inefficiency  often  is 
difficult  to  measure  because  it  requires  that  one  determine  the 
characteristics  of  efficient  problem  solutions  for  Infrequent,  ill-defined, 
and/or  multi-event  situations.  Overall,  considering  the  trade-off  between 
ease  of  measurement  and  richness  of  information,  human  error  is  probably  the 
most  interesting  dimension  of  naintalner  performance. 

Human  Error 

Human  error  is  a  topic  of  great  interest  to  many  people  ranging  from 
pyschoanalysts  to  psychologists  to  reliability  engineers.  A  wide  variety  of 
theories  and  classification  schemes  have  been  proposed  and.  In  a  few  cases, 
evaluated.  Recently,  a  comprehensive  methodology  for  analysis  and 
classification  of  human  errors  has  been  proposed  and  applied  to  studies  of 
human  error  in  three  fairly  different  domains  (Rouse  &  Rouse,  1982). 

For  the  purposes  of  this  paper,  the  main  interest  in  human  error  is  as  a 
performance  measure  suitable  for  closing  the  loops  shown  in  Figure  2. 

Further,  error  has  a  particularly  attractive  feature  In  that  its  desired  level 
can  be  set  a  priori  as  zero.  (In  this  respect,  time  and  inefficiency  are  not 
so  straightforward;  zero  time  and  inefficiency  may  be  desirable,  but  not 
realistically  achievable.)  Thus,  human  error  inherently  provides  a  definition 
of  desired  maintainer  performance  as  well  as  a  means  of  feeding  back  actual 
maintainer  performance.  The  next  question  is  whether  or  not  this  results  in 
the  process  depicted  in  Figure  2  being  able  to  adapt. 

Such  adaptation  Is  possible.  However,  it  requires  that  human  error  be 
analyzed  and  classified  at  a  fairly  fine-grained  level.  Rouse  and  Rouse 
(1982)  have  proposed  a  classification  scheme  Involving  6  general  and  31 
specific  categories  of  human  error.  The  general  categories  Include  human 
error  related  to: 

1.  Observation  of  system  state 

2.  Choice  of  hypothesis 

3.  Testing  of  hypothesis 

4.  Choice  of  goal 

5.  Choice  of  procedure 

fi.  Extension  of  procedure. 

(The  specific  categories  are  too  numerous  to  list  and  define  within  this 
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paper.)  The  sources  of  data  used  to  identify  and  classify  human  error  include 
observer  notes,  questionnaires,  and  interviews  as  well  as  recordings  of  system 
state  variables,  communications  among  personnel,  and  occasionally  verbal 
protocols  (i.e.,  "thinking  aloud"). 

The  methodology  has  been  applied  within  three  studies.  Within  a  study  of 
aircraft  mechanics,  the  methodology  was  used  to  identify  a  training  deficiency 
and  subsequently  evaluate  an  improved  training  program  (Johnson  A  Rouse, 

1982).  For  a  study  of  aircraft  pilots,  the  methodology  was  employed  to 
identify  the  benefits  of  a  computer-based  display  system  in  terms  of  a 
substantial  decrease  in  the  frequencies  of  certain  types  of  error  (Rouse, 
Rouse,  A  Hammer,  1982;  Rouse  A  Rouse,  1982).  Within  a  study  of  supertanker 
engineering  officers,  the  methodology  was  utilized  to  identify  deficiencies  in 
the  design  of  the  control  panel  and  inadequacies  in  the  operator' s  knowledge 
of  system  functions  which  led  to  a  plan  to  modify  the  training  program  (van 
Fekhout  A  Rouse,  1981).  Taken  as  a  whole,  these  three  studies  show  that 
defining  performance  appropriately  can  lead  to  the  type  of  adaptations 
represented  in  Figures  1  and  2. 

Conclusions 

This  paper  has  proposed  that  training  is  one  of  the  most  responsive  means 
available  for  adapting  to  meet  performance  objectives.  It  was  suggested  that 
suitable  definitions  of  desired  performance  and  measures  of  actual  performance 
are  needed  in  order  for  the  potential  adaptivity  of  training  to  be  realized. 
The  use  of  human  error  to  provide  the  necessary  measures  of  desired  and  actual 
performance  was  discussed  and  results  of  utilizing  these  measures  briefly 
reviewed. 

The  implications  of  the  point  of  view  espoused  in  this  paper  go  beyond 
analysis  and  classification  of  human  error.  The  broadest  and  most  important 
Implication  is  that  feedback  is  necessary  if  maintenance  (or  operational) 
problems  are  to  be  lessened  or  eliminated.  While  the  feedback  provided  by 
existing  maintenance  record  systems  (i.e.,  3-M,  TAWS,  and  66-1/66-5)  may  be 
sufficient  as  a  warning  device  for  noting  the  presence  of  problems,  it  is 
insufficient  for  isolating  the  sources  of  problems.  Once  problems  are 
detected,  a  more  fine-grained  analysis  is  necessary.  Thus,  one  can  envision 
there  being  at  least  two  levels  of  feedback;  one  for  detecting  problems  and 
one  for  Isolating  causes  and  adapting  appropriately. 
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This  paper  compares  the  cost-effectiveness  of  maintenance  training 
simulators  and  actual  equipment  trainers  for  use  in  training  military 
personnel  how  to  maintain  operational  equipment.  Both  types  of  equipment  have 
been  used  for  training  personnel  to  perform  corrective  and  preventive 
maintenance  at  organizational  and  intermediate  levels. 

An  actual  equipment  trainer  is  simply  a  unit  of  operational  equipment 
brought  into  a  «chool  for  training  purposes.  It  may  or  may  not  be  modified  to 
make  it  operate  in  a  classroom  and  to  be  more  convenient  for  training 
purposes.  A  maintenance  training  simulator  is  a  device  that,  in  some  way, 
mimics  operational  equipment  for  use  in  training.  In  recent  years,  there  has 
been  a  trend  to  use  maintenance  training  simulators  rather  than  actual 
equipment  for  training  purposes.  Maintenance  simulators  are  said  to  have 
advantages  over  actual  equipment  for  use  in  training  such  as  lower  cost, 
ability  to  demonstrate  a  wider  variety  of  malfunctions  and  more  freedom  from 
breakdown  in  the  classroom. 

Characteristics  of  Maintenance  Simulators 

Maintenance  simulators  differ  in  how  closely  they  resemble  actual 
equipment,  in  their  functional  capabilities  as  instructional  devices,  and  in 
their  complexity  and  cost.  These  simulators  are  often  characterized  as  2-D  or 
3-D  devices,  i.e.,  as  being  two-  or  three-dimensional  in  their  physical  form; 
some  simulators  contain  both  2-D  and  3-D  components. 

The  manufacturers  of  2-D  simulators  have  developed  software  packages, 
computer  and  support  components  that  can  be  used  in  a  number  of  different 
simulations.  This  has  led  us  to  distinguish  between,  what  we  call  later  in 
discussing  costs,  "standard"  and  "nonstandard"  maintenance  simulator  systems. 
Standard  systems,  whether  they  are  2-D  or  3-D  simulators,  are  likely  to  cost 
less  than  nonstandard  systems.  A  3-D  simulator  permits  "hands  on"  practice  in 
manual  maintenance  skills  not  possible  on  many  2-D  simulators;  it  also  has 
greater  physical  similarity  to  the  actual  equipment.  Whether  or  not  greater 


physical  similarity  increases  the  effectiveness  of  training  is  a  proper 
question. 

Advantages  of  Maintenance  Simulators 

The  major  advantage  of  a  maintenance  simulator  is  that  it  is  designed  to 
be  used  as  a  training  device  and  to  provide  facilities  important  for 
instructing  students.  In  contrast,  actual  equipment  is  designed  to  operate 
effectively  under  a  variety  of  stressful  conditions  in  the  field;  it  is  not 
meant  to  be  used  as  a  training  device. 

Maintenance  simulators  can  be  designed  to  include  a  large  variety  of 
malfunctions  with  which  maintenance  personnel  should  be  familiar,  including 
faults  that  cannot  be  demonstrated  conveniently  on  actual  equipment  trainers 
or  that  occur  rarely  in  real  life.  All  modern  maintenance  simulators 
incorporate  some  type  of  computer  support.  Thus,  the  symptoms  of  many  types 
of  complex  faults  can  be  stored  in  the  computer  and  selected  simply  by  a 
control  setting  on  the  instructor' s  console.  Computer-supported  equipment  can 
also  record  what  the  student  does,  thereby  reducing  the  need  for  constant 
observation  by  the  instructor.  The  instructor  can  use  information  collected 
by  the  computer  to  guide  each  student;  a  computer  can  also  assist  the  student 
without  an  instructor's  intervention.  Records  of  student  performance  and 
achievement  can  be  maintained  automatically.  Simulators  can  be  made  rugged 
enough  to  sustain  damage  or  abuse  encountered  from  students.  Thus,  they  can 
provide  greater  reliability  and  availability  in  the  classroom  than  is  often 
found  with  actual  equipment.  Training  which  would  be  avoided  because  of 
safety  reasons,  e.g.,  exposure  of  students  to  dangerous  electrical  currents  or 
hydraulic  pressures,  can  be  undertaken  with  little  risk  with  a  simulator.  If 
students  using  such  equipment  complete  their  training  in  less  time,  as  has 
often  been  found  with  computer-based  methods  of  instruction,  there  are 
potential  cost  reductions  due  to  savings  in  student  time,  increased  throughput 
of  students  and  reduced  need  for  instructors  and  support  personnel. 

A  simulator  need  not  contain  all  the  components  found  in  the  actual 
equipment.  Thus,  it  is  often  possible  to  build  a  simulator  that  has  greater 
flexibility  and  capacity  for  training  and  that  costs  less  than  an  actual 
equipment  trainer. 

Disadvantages  of  Maintenance  Simulators 

There  are  some  disadvantages  to  the  use  of  simulators.  The  procurement  of 
maintenance  simulators  necessarily  involves  some  costs  to  design  and  build 
this  special  equipment,  to  develop  course  materials,  maintenance  procedures, 
support  and  documentation.  The  types  of  training  provided  by  simulators  may 
not  provide  the  student  with  all  the  skills  needed  to  maintain  operational 
equipment,  an  outcome  that  seems  assured  when  actual  equipment  is  used  for 
training.  A  new  simulator  may  not  be  ready  when  needed  for  training  because 
its  design  and  development  necessarily  follows  that  of  the  actual  equipment; 
modifications  in  the  design  of  the  actual  equipment  may  delay  completion  of 
the  simulator,  which  must  be  modified  accordingly.  If  there  are  many  and 
frequent  modifications,  the  original  simulator  may  have  to  be  redesigned 
totally,  sometimes  at  a  large  cost,  in  order  to  be  useful  for  training. 


Data  on  the  effectiveness  and  cost  of  maintenance  simulators  and  actual 
equipment  trainers  are  considered  next. 

The  Effectiveness  of  Maintenance  Simulators 


The  purpose  of  maintenance  training,  whether  with  simulators  or  actual 
equipment  trainers,  is  to  qualify  technicians  to  maintain  equipment  in  the 
field.  In  fact,  however,  the  effectiveness  of  maintenance  simulators  for 
training  technicians  has  been  compared  to  that  or  actual  equipment  trainers 
only  on  the  basis  of  student  performance  at  school  and  not  on  the  job;  an 
exception  to  this  general  statement  (Cicchinelli ,  Harmon,  Keller  4 
Kottenstette,  1980)  will  be  discussed  later.  The  lack  of  job  performance  data 
to  validate  training  applies  generally  to  all  types  of  military  training  and 
not  to  maintenance  training  alone. 

Effectiveness  of  Maintenance  Simulators  at  Schools 


We  found  12  studies,  conducted  over  the  period  of  1967  to  1980,  that 
compare  the  effectiveness  of  maintenance  simulators  and  actual  equipment 
trainers  in  a  variety  of  courses  at  military  training  schools;  these  are 
sumnari.  jd  in  Table  1.  Most  of  the  maintenance  simulators  apply  to 
electronics  and  aviation;  one,  the  Hagen  Automatic  Boiler  Control,  involves  an 
electro-mechanical  control  system  for  ships. 

Student  Achievement 


Effectiveness  was  evaluated  by  comparing  the  scores,  in  end-of-course 
tests,  of  students  who  used  simulators  with  those  who  used  actual  equipment 
trainers.  There  are  13  comparisons;  in  12  of  these,  students  trained  with 
simulators  achieved  test  scores  the  same  as  or  better  than  those  trained  with 
actual  equipment;  in  one  case,  scores  were  lower.  The  differences,  though 
statistically  significant,  have  little  practical  significance. 

Cicchinelli  et  al .  (1980)  compared  supervisors'  ratings  on  the  job 
performance  of  technicians  trained  either  with  a  maintenance  simulator  (the 
6883  Test  Station  3-D  Simulator)  or  an  actual  equipment  trainer.  Field 
surveys  provided  ratings  on  the  job  performance  of  course  graduates  (some 
twice);  some  were  on  the  job  as  long  as  32  weeks.  The  supervisors  did  not 
know  how  the  students  had  been  trained.  Their  ratings  showed  no  noticeable 
difference  between  the  performance  of  technicians  trained  with  the  simulator 
or  with  the  actual  equipment  trainer.  The  abilities  of  the  technicians  in 
both  groups  increased  with  amount  of  time  on  the  job. 

Time  Savings 

The  automated  and  individualized  method  of  instruction  that  is  an  inherent 
characteristic  of  modern  maintenance  simulators  should  be  expected  to  save 
some  of  the  time  students  need  to  complete  the  same  course  when  given  by 
conventional  Instruction  (Orlansky  4  String,  1979).  Time  savings  are  reported 
in  three  of  the  studies  shown  in  Table  1.  Compared  to  the  use  of  actual 
equipment  trainers,  maintenance  simulators  saved  22,  50  and  50  percent, 
respectively ,  of  the  time  students  needed  to  complete  these  courses.  Although 
no  explanations  are  offered  for  these  time  savings,  one  could  surmise  that 


*Orlansky  and  String,  1981 


they  are  due  to  factors  such  as  that  brighter  students  can  complete  a 
self-paced  course  faster  than  one  given  by  conventional,  group-paced 
Instruction,  that  maintenance  simulators  generally  have  greater  availability 
in  the  classroom  than  do  actual  equipment  trainers  and  that  instructors  need 
less  time  to  set  up  training  problems  and/or  to  insert  malfunctions  in 
simulators  than  in  actual  equipment  trainers. 

Attitudes 

Based  on  questionnaires  administered  at  the  completion  of  the  courses, 
students  favor  the  use  of  simulators  in  9  out  of  10  cases  and  are  neutral  in 
one.  Instructors  are  divided  evenly  as  being  favorable,  neutral  or 
unfavorable  to  the  use  of  simulators  (about  one-third  each). 

Conclusions 

Overall,  maintenance  simulators  appear  to  be  as  effective  as  actual 
equipment  trainers  for  training  military  personnel  at  schools;  there  is  only 
one  contrary  finding.  The  effect  of  training  upon  job  performance  is  reported 
only  by  Cicchinelli  et  al.  (1980),  who  found  no  difference  between  those 
trained  with  a  simulator  and  those  with  an  actual  equipment  trainer. 

The  Cost  of  Maintenance  Simulators 

In  order  to  deal  with  the  issue  of  costs,  we  divided  simulators  into  three 
classes. 

Standard  Systems 

This  class  of  maintenance  simulators  is  based  on  standardization  of  the 
physical  configuration.  Such  simulators  consist  of  two  elements:  one 
element,  called  here  the  "general  simulation  system"  constitutes  a  generalized 
and  adaptable  (but  incomplete)  simulation  capability  that  can  satisfy  a  wide 
range  of  specific  training  applications.  The  second  element  tailors  the 
general  simulation  system  to  a  particular  training  application;  it  is 
typically  limited  to  courseware  and  pictorial  or  other  representations  (i.e., 
the  simulation  model)  of  the  particular  equipment  being  simulated.  Standard 
systems  were  the  earliest  type  to  be  used  for  maintenance  training  and  are  the 
only  class  to  achieve  extensive  use.  Compared  with  the  other  classes  of 
simulators,  the  standard  systems  are  generally  low  in  cost  and  limited  in 
terms  of  the  complexity  of  processes  that  can  be  simulated.  About  650  units 
of  standard  simulators  have  been  procured  for  about  200  different  training 
applications  (many  produced  by  ECC,  Burtek,  Ridgeway,  and  Lockheed). 

Nonstandard  Systems 

The  outstanding  characteristic  of  nonstandard  systems  is  diversity  with 
respect  to  contractors  and  types  of  contracts,  program  purpose,  numbers  of 
devices  manufactured,  physical  characteristics,  complexity,  and  cost.  The 
physical  characteristics  of  the  nonstandard  simulators  vary  widely  and  include 
two-  and  three-dimensional  trainers.  There  Is  wide  variability  in  the 
software.  Further,  since  most  nonstandard  systems  typically  simulate  only  one 
operational  system,  there  is  no  definitive  separation  between  software  and 


courseware  functions.  At  the  tine  of  writing,  there  were  17  nonstandard 
naintenance  simulator  programs  that  should  produce  47  unique  simulations 
(e.g.,  the  Mk  92  Fire  Control  System,  Close-In  Weapon  System,  F-16,  MA-3,  and 
6883  Test  Bench)  and  the  delivery  of  687  units,  i.e.,  individual  trainers. 
Producers  of  these  simulators  include  Honeywell,  Vought,  AppH-Mation, 

Grumman,  and  RCA. 

CAI-Like  Systems 

A  CAI-like  maintenance  simulator  is  a  computer-assisted  instruction  (CAI) 
system  with  courseware  designed  specifically  to  train  maintenance  skills.  A 
typical  CAI  system  uses  two-dimensional  displays  (cathode  ray  tube  and/or 
random  access  slides,  microfiche,  or  videodisc  to  present  lesson  materials, 
pictures  of  equipment  and  the  like)  under  control  of  a  computer  that  also 
monitors  student  progress,  prescribes  lessons,  and  scores  tests.  When  adapted 
to  maintenance  training,  the  CAI  features  are  retained,  and  the  trainer  may 
also  employ  three-dimensional  versions  of  equipment.  Examples  of  such  systems 
are  the  Navy  Electronic  Equipment  Maintenance  Trainer  and  the  Army  Maintenance 
Training  and  Evaluation  Simulation  System.  Insufficient  cost  data  were 
available  on  CAI-like  maintenance  simulators  and  they  are  not  discussed 
further. 

Costs  of  Maintenance  Training  Simulators 

We  found  that  the  data  now  available  on  standard  systems  are  insufficient 
to  analyze  their  elements  of  cost  and  to  relate  these  cost  elements  to  the 
physical  and  performance  characteristics  of  the  trainers.  In  effect,  it  is 
now  difficult  or  impossible  to  Identify  the  major  cost  distinctions  (e.g., 
between  recurring  and  nonrecurring  costs,  between  development  and  fabrication, 
betv/een  hardware  and  software)  that  allow  characteristics  of  the  simulator  to 
be  related  to  the  total  cost  of  the  simulator  program. 

Data  from  nine  contracts  for  standard  simulators  were  reviewed,  and  the 
Information  they  contain  is  shown  In  Figure  1.  These  contracts  involve  the 
development  of  67  different  models  of  simulators  and  the  delivery  of  a  total 
of  444  units.  The  figure  shows  average  contract  cost  per  delivery  (total 
contract  value  divided  by  the  number  of  trainers  procured)  vs.  the  number  of 
trainers  procured  in  each  contract.  These  simulators  ranged  in  unit  cost  from 
about  $10  thousand  to  $204  thousand  each  with  a  median  cost  of  about  $33 
thousand.  The  unit  cost  is  reduced  as  the  number  of  units  in  each  contract 
Increases.  However,  caution  is  advised  in  using  the  data  in  this  figure.  The 
individual  contracts  involve  varying  numbers  of  simulation  models  as  well  as 
quantities  of  trainers;  both  the  simulation  models  and  the  trainers  vary  in 
complexity,  physical,  and  performance  characteristics. 

The  cost  of  13  nonstandard  maintenance  simulators  is  shown  in  Table  2. 

The  estimates  are  normalized  to  show  recurring  production  costs  adjusted  to 
reflect  a  production  quantity  of  one.  These  simulators  range  in  cost  from 
$100  thousand  to  $4.5  million;  the  median  value  is  $900  thousand. 

Nonrecurring  costs  account  for  a  large  portion  of  the  total  program  costs 
of  nonstandard  maintenance  s1mulators--over  70  percent  when  only  one  unit  is 
fabricated  and  about  50  percent  when  five  or  six  are  fabricated  (Figure  2). 
Software  and  courseware  account  for  10  to  45  percent  of  total  program  costs 
(Figure  3). 
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Table  2 

Nonstandard  Maintenance  Simulators:  Cost  of  13 
Trainers  (Normalized  to  Production  of  One  Unit) 


TRAINER 

COST 
$  (000) 

AN/TPS-43  GROUND  RADAR 

$  100 

TRIDENT  AIR  CONDITIONER 

135 

TRIDENT  HIGH  PRESSURE  AIR  COMPRESSOR 

140 

F-111D  AVIONICS  TEST  BENCH  (2-D  6883) 

395 

A-6E  TRAM 

475 

MA-3  GENERATOR/CONSTANT  SPEED  DRIVE 

525 

TEST  STAND 

AWACS  RADAR  SYSTEM 

900 

F-1I1D  AVIONICS  TEST  BENCH  (3-D  6883) 

920 

A-7E  HEADS-UP  DISPLAY  TEST  BENCH 

1295 

F-4J/N  (AT  TRAINER) 

1540 

AWACS  NAVIGATION/GUIDANCE  SYSTEM 

2460 

TRIDENT  INTEGRATED  RADIO  ROOM  -  MAINTENANCE 

2625 

TRAINER 

TRIDENT  INTEGRATED  RADIO  ROOM  -  OPERATOR/ 

4465 

MAINTENANCE  TRAINER 

>.v ■< 
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QUANTITY  FABRICATED 

KEY:  IRR-1,  TRIDENT  maintenance  trainer 

IRR-2,  TRtOENT  operations/maintenance  trainer 
▲  A-6E  DRS,  excluding  engineering  change 
*  Excludes  evaluation  costs 

Figure  2.  Nonstandard  maintenance  simulators:  nonrecurring  cost  as  a  percent 
of  program  cost,  according  to  quantity  fabricated. 


QUANTITY  FABRICATED 


KEY:  IRR-1,  TRIDENT  maintenance  trainer 

IRR-2,  TRIDENT  operations/maintenance  trainer 
▲  A-6E  DRS,  excluding  engineering  changes 
*  Excludes  evaluation  costs 

Figure  3.  Nonstandard  maintenance  simulators:  software  and  courseware  cost 
as  a  percent  of  program  total  cost,  according  to  quantity 

fabricated. 


Cost-Effectiveness  of  Maintenance  Simulators 

Student  achievement  at  school,  as  reported  above,  is  about  the  same 
whether  students  are  trained  with  maintenance  simulators  or  with  actual 
equipment  trainers.  Therefore,  the  cost-effectiveness  of  maintenance 
simulators  depends  on  how  much  they  cost,  compared  to  the  cost  of  actual 
equipment  trainers.  There  is  one  comparison  of  life-cycle  costs  that  we  will 
consider  separately.  The  cost  comparisons  that  follow  are  incomplete  because 
they  include  only  acquisition  costs. 

The  cost  of  an  actual  equipment  trainer  is  the  recurring  production  cost 
of  an  additional  unit  of  equipment,  under  procurement  as  part  of  some  weapon 
system,  and  does  not  include  any  costs  of  research,  development,  test  and 
evaluation  (RDT&E).  Adapting  a  production  unit  for  use  in  training,  such  as 
by  adding  power,  special  inputs  and  controls,  may  require  some  additional 
costs. 

We  were  able  to  get  relatively  complete  data,  useful  for  comparative 
purposes,  on  both  maintenance  simulators  and  actual  equipment  trainers,  for  11 
cases;  actual  equipment  trainers  had  not  been  used  previously  in  some  cases 
where  maintenance  simulators  were  developed  recently  for  training.  Some  of 
the  simulators  are  prototypes,  rather  than  production  units.  Data  on  these 
simulators  Include  the  costs  of  RDTSE.  These  should  be  removed  in  order  to 
make  a  fair  comparison  with  the  cost  of  actual  equipment  trainers  which,  as 
noted  above,  exclude  the  costs  of  RDTSE.  The  number  of  maintenance  simulators 
procured  could  also  influence  the  cost  of  a  single  unit;  this  varied  from  1  to 
36. 

We  decided  to  use  values  which  would  provide  high  and  low  estimates  for 
the  cost  of  one  maintenance  simulator.  These  were: 

High  cost  estimates.  Total  program  costs  adjusted  to  reflect  a  production 
quantity  of  one;  this  includes  the  nonrecurring  costs  of  research  and 
development  but  not  of  test  and  evaluation  and  the  recurring  production 
cost  of  one  unit.  We  call  this  the  "Simulator  Normalized  Total  Program 
Cost." 

Low  cost  estimate.  The  recurring  fabrication  cost  of  one  follow-on 
maintenance  simulator.  We  call  this  the  "Simulator  Unit  Recurring 
Fabrication  Cost." 

The  high  cost  estimates  are  shown  in  Figure  4.  The  ratio  of 
sinulator/actual  equipment  trainer  costs  is  0.60  or  less  for  seven  cases 
(range  0.25  to  0.55).  There  are  four  cases  where  this  ratio  varies  from  1.60 
to  4.00  (VTAS,  MA-3,  AT  Trainer  and  AWACS).  We  believe  these  data  are  suspect 
for  the  following  reasons:  in  two  cases,  the  operational  equipments  are 
relatively  old  and  their  costs  have  not  been  adjusted  to  reflect  replacement 
values;  in  two  cases,  the  simulator  programs  incurred  costs  for  tasks  beyond 
simply  developing  a  simulator  for  routine  training.  For  these  reasons,  we 
decided  to  accept  0.60  as  an  upper  limit  for  the  relative  cost  of  a 
maintenance  simulator  compared  to  an  actual  equipment  trainer. 

The  low  cost  estimates,  based  on  the  recurring  cost  of  these  simulators, 
are  shown  in  Figure  5.  Nine  of  the  11  cases  fall  at  0.20  or  lower;  the  range 


Is  0.03  to  0.19.  The  two  outliers  (VTAS  and  HA-3)  are  regarded  as  atypical 
for  the  reasons  set  forth  above. 

The  cost-effectiveness  of  a  maintenance  simulator  on  a  life-cycle  basis 
has  been  evaluated  only  in  one  case  that  compares  the  Air  Force  6883  Test 
Stand  3-Dimensional  Simulator  with  the  6883  Actual  Equipment  Trainer 
(Clcchinelli  et  al.,  1980).  The  three-dimensional  simulator  and  actual 
equipment  trainer  were  equally  effective  when  measured  by  student  achievement 
at  school;  supervisors'  ratings  showed  no  difference  between  the  job 
performance  of  students  trained  either  way  for  periods  up  to  32  weeks  of 
experience  after  leaving  school. 

The  life-cycle  cost  comparison  of  simulator  and  actual  equipment  trainer 
is  shown  in  Table  3.  Costs  v/ere  estimated  in  constant  1978  dollars  over  a 
15-year  period  and  discounted  at  10  percent.  The  results  show  that  the  total 
cost  per  student-hour  was  $23  for  the  simulator  and  $60  for  the  actual 
equipment  trainer,  i.e.,  38  percent  as  much  for  the  simulator,  compared  to  the 
actual  equipment  trainer,  for  all  costs  over  a  15-year  period.  The  simulator 
costs  less  to  procure  ($595  thousand  vs.  $2105  thousand,  or  28  percent  as 
much)  and  less  to  operate  ($1588  thousand  vs.  $3367  thousand  or  47  percent  as 
much)  over  a  15-year  period. 

We  draw  the  following  conclusions: 

Cost.  Maintenance  simulators  cost  less  to  procure  than  do  actual 
equipment  trainers,  i.e.,  20  to  60  percent  as  much,  using  the  low  and  high 
cost  estimates,  and  less  on  a  life-cycle  basis. 

Effectiveness.  Achievement  at  school  is  the  same  when  students  are 
trained  with  nai ntenance  simulators  or  with  actual  equipment  trainers. 

This  finding  applies  to  12  out  of  13  cases  in  which  such  comparisons  were 
made. 

Table  3 


6883  Test  Stand:  15-Year  Life-Cycle  Costs  of  Actual 
Equipment  Trainer  and  3-Dimensional  Simulator* 


ITEM 

(THOUSANDS  OF  DOLLARS) 

ACTUAL  3-D 

EQUIPMENT  SIMULATOR 

SIMULATOR/ 

AET(*) 

ACQUISITION 

$2105 

$  595 

28% 

RECURRING  COSTS 

3367 

1588 

47 

TOTAL 

5472 

2183 

40 

NET  PRESENT  VALUE 
(1978  DOLLARS) 

3896 

1501 

39 

COST  PER  STUDENT 

HOUR 

60 

23 

38 

♦Chicchinelli,  Harmon,  Keller,  Kottenstette,  1980 


Di scussion 


Maintenance  simulators  are  cost-effective  compared  to  actual  equipment  •') 

trainers. 

This  finding  is  necessarily  qualified  by  the  limited  nature  of  the  data 
from  which  it  is  derived.  Effectiveness,  as  used  here,  is  based  on 
performance  demonstrated  at  school  rather  than  on  the  job.  Cost,  in  all  but 
one  case,  refers  to  the  initial  costs  of  acquiring  training  equipment  and  does 
not  include  the  costs  associated  with  the  long-term  use  of  simulators  or  of  Cj 

actual  equipment  for  training,  e.g.,  maintenance  and  upkeep,  instructors  and 
support  personnel ,  student  pay  and  support.  In  the  one  case  wh ere  a 
life-cycle  cost  comparison  was  made,  total  cost  per  student-hour  over  a 
15-year  period  for  the  6883  Test  Stand  3-Dimensional  Simulator  was  38  percent 
as  much  as  for  the  actual  equipment  trainer.  Both  were  equally  effective  as 
measured  by  tests  at  school  and  by  supervisors'  ratings  of  performance  on  the 
job  after  leaving  school. 


Conclusions 


1.  Maintenance  simulators  are  as  effective  as  actual  equipment  trainers 
for  training  military  personnel,  as  measured  by  student  achievement  at  school 
and,  in  one  case,  on  the  job.  The  use  of  maintenance  simulators  saves  some  of 
the  time  needed  by  students  to  complete  courses,  but  data  on  this  point  are 
limited.  Students  favor  the  use  of  maintenance  simulators;  instructors  are 
favorable,  neutral,  or  negative  to  the  use  of  simulators  in  about  equal 
amounts. 

2.  The  acquisition  cost  of  maintenance  simulators  is  typically  20  or  60 
percent  that  of  actual  equipment  trainers,  depending  upon  whether  simulator 
program  nonrecurring  costs  are  included  or  not.  One  life-cycle  cost  estimate 
shows  that  purchase  and  use  of  a  simulator  would  cost  38  percent  as  much  over 
a  15-year  period  as  it  would  for  an  actual  equipment  trainer. 

3.  Maintenance  simulators  are  as  effective  as  actual  equipment  trainers 
for  training  maintenance  personnel.  They  cost  less  to  acquire.  Therefore, 
maintenance  simulators  are  cost-effective  compared  to  actual  equipment 
trainers. 

4.  The  data  on  the  cost  and  effectiveness  of  maintenance  simulators  have 
not  been  collected  in  a  systematic  manner.  Therefore,  there  is  no  basis  at 
present  for  making  trade-offs  between  the  effectiveness  and  cost  of  different 
types  of  maintenance  simulators  on  such  issues  as  two-dimensional  vs. 
three-dimensional  design,  the  complexity  of  maintenance  simulators  (in  such 
terms  as  number  of  malfunctions  and  instructional  procedures),  the  extent  to 
which  simulators  should  provide  a  mixture  of  training  in  general  maintenance 
procedures  and/or  for  maintaining  specific  equipment,  and  the  optimum 
combination  of  maintenance  simulators  and  actual  equipment  trainers  for 
training  technicians  at  school. 

5.  The  conclusions  to  this  paper  must  be  qualified  by  the  fact  that  they 
are  based  on  limited  and  often  incomplete  data.  There  is  a  need  for  data 
suitable  for  analyses  if  comparisons  of  maintenance  simulators  and  actual 
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equipment  trainers  are  to  be  made  in  the  following  areas:  life-cycle  costs, 
on-the-job  performance,  and  student  attrition  at  school.  There  is  also  a  need 
for  data  if  cost-effectiveness  comparisons  are  to  be  made  among  simulators 
that  vary  in  complexity  of  design,  e.g.,  two-  and  three-dimensional  simulators 
and  types  of  instructional  features. 
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Biodynamic  Modeling  of  the  Maintainer 


E.  R.  Winkler 
J.  T.  Miller 


McDonnell  Douglas  Corporation 
St.  Louis,  Missouri 


The  Maintenance  Action  Model  (MAM),  McBride  and  Lambert  (1982),  is  one 
component  of  the  U.S.  Navy  Design-for-Maintainers  program.  The  MAM  consists 
of  a  set  of  conceptual  tools  that  can  be  used  by  designers  to  ensure  that 
man-machine  interfaces  are  considered  in  all  phases  of  equipment  design  from 
conceptualization  to  retrofit.  One  requirement  of  the  MAM  is  the 
incorporation  of  a  biodynamic  strength  model. 

The  desire  of  Industry,  largely  sponsored  by  the  National  Institute  for 
Occupational  Safety  and  Health  (NIOSH),  to  achieve  a  safer  workplace  for 
individuals  handling  materials  has  greatly  accelerated  the  recent  development 
and  validation  of  biodynamic  strength  models.  The  development  and 
incorporation  of  these  types  of  models  into  the  maintenance  design  process 
would  complement  and  extend  current  design  guidelines  such  as  Military 
Standard  1472C.  Specifically,  a  biodynamic  strength  model  could  extend 
current  strength  limitation  guidelines  to  account  for: 
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•  the  female  population 

•  a  larger  number  of  body  positions  than  are  defined  by  current 

guidel ines 

•  a  mechanism  for  defining  limits  in  design-specific  or  restricted 

workspaces 

t  tasks  which  require  extended  exertion,  i.e.,  define  fatigue  limits 

•  tasks  which  require  repetitive  actions 

•  a  mechanism  for  defining  limits  for  unique  or  design-specific  body 

positions 

t  tasks  with  dynamic  as  well  as  static  (isometric)  requirements. 

Applications  and  Existing  Models 

Experience  with  the  F/A-18,  F-4,  AV-8B,  and  MDC  missile  systems  indicates 
that  a  biodynamic  strength  model  could  have  significant  impact  In  a  number  of 
areas.  A  primary  consideration  is  the  need  to  acquire  the  fifth  and 
ninety-fifth  percentile  strength  limitations  for  a  wide  variety  of  body 
positions.  A  usable  biodynamic  model  could  provide  this  Information  In  a 
timely  and  cost-effective  manner  by  eliminating  the  need  to  construct  mockups 
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or  simulators.  This  could  be  directly  applicable  to  VTX.  It  is  especially 
important  that  this  information  be  available  for  the  female  population,  since 
more  females  will  be  employed  in  maintenance  specialities  in  the  future 
(Report  NRAC  80-9). 

Our  experience  has  demonstrated  a  requirement  for  a  biodynamic  strength 
model  that  can  be  used  to  predict  strength  limitations  when  design 
considerations  dictate  a  restricted  workspace  which  may  force  maintenance 
personnel  to  assume  unique  body  positions.  Restricted  workspaces  may  also 
require  maintainers  to  move  or  lift  materials  through  nonoptimal  paths. 

We  have  also  found  that  a  biodynamic  strength  model  must  either 
incorporate,  or  accommodate  the  incorporation  of,  procedures  for  predicting 
the  limits  of  fatigue  when  various  repetitive  tasks  are  performed.  These 
problems  have  been  most  frequently  identified  in  the  test  and  evaluation  pnase 
of  the  F/A-18  program. 

Additionally,  there  is  a  need  for  a  biodynamic  strength  model  which  can 
predict  strength  limitations  for  various  hand  functions.  Specifically, 
prediction  techniques  for  grasp  strength,  twisting  strength  (either  with  the 
hand  or  just  the  digits),  and  whole  hand  or  digit  pushing  and  pulling  strength 
limitations  are  required.  The  whole-body  biodynamic  strength  model  must 
either  incorporate,  or  be  easi ly  modified  to  incorporate,  hand  strength 
prediction  features. 

Recent  requirements  of  various  industries  with  extensive  materials- 
handling  occupations  have  resulted  in  the  development  of  a  number  of 
biodynamic  strength  models.  We  reviewed  those  models  to  determine  if  any 
could  be  adapted  for  use  with  a  weapon  system  like  the  F/A-18.  Three 
different  modeling  approaches  have  been  used--physiological ,  psychophysical, 
and  biomechanical.  These  models  have  been  developed  from  basic  precepts  that 
vary  widely,  and  this  has  influenced  the  domain  of  strength-related  questions 
to  which  they  have  been  applied. 

Physiological  models  have  been  concerned  primarily  with  repetitive  tasks 
and  the  levels  of  exertion  that  result  in  various  levels  of  fatigue.  Fatigue 
effects  are  generally  reflected  in  changes  in  the  rate  of  oxygen  consumption, 
heart  rate,  and  rate  of  metabolic  energy  expenditure. 

The  psychophysical  approach  is  primarily  empirical.  Individuals  are 
required  to  lift,  lower,  push,  pull,  or  carry  objects  based  on  nominal  or 
standard  body  positions  (postures).  Voluntary  load  limits  are  obtained  and 
are  incorporated  into  a  data  base  which  can  be  analyzed  using  multiple 
regression  techniques.  These  analyses  generate  a  model  in  which  various 
physical  characteristics  and  voluntary  forces  are  entered  as  a  basis  for 
strength  predictions. 

Biomechanical  models  have  been  developed  to  examine  the  forces  that  are 
exerted  on  various  body  components  when  the  body  is  treated  as  a  purely 
mechanical  structure.  This  type  of  analysis  examines  each  body  segment  which 
includes  the  joints,  muscle  groups,  and  centers  of  mass,  and  then  computes  the 
forces  and  moments  that  result  from  loads  imposed  on  the  body.  Strength 
limitations  are  defined  on  the  basis  of  the  structural  damage  which  would 
occur  if  various  load  limits  were  exceeded. 


Evaluation  of  Current  Models 
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Survey  Methodology 


An  initial  survey  of  the  literature  on  strength  modeling  indicated  that 
three  major  university  laboratories  are  working  in  this  area,  specifically: 

•  University  of  Illinois--Chicago  Circle  (Illinois),  physiological 

model i ng 

•  Texas  Technological  University  (Texas),  psychophysical  modeling 

•  University  of  Michigan  (Michigan),  biomechanical  modeling. 

A  questionnaire  addressing  major  issues  related  to  biodynamic  strength  model 
requirements  was  prepared.  Questions  concerned  with  the  structure  and 
properties  of  the  models  were  developed  on  the  basis  of  the  preliminary  search 
of  the  literature.  Questions  concerned  with  the  user  requirements  were 
developed  with  the  assistance  of  human  factors  specialists  responsible  for 
maintenance  design  on  a  variety  of  aircraft  and  missile  programs  (F/A-18, 

F-15,  AV-8B,  Harpoon).  The  questionnaire  was  forwarded  to  cognizant 
individuals  at  the  Illinois,  Texas,  and  Michigan  laboratories. 

We  then  visited  each  of  the  laboratories  and  interviewed  model  developers, 
individuals  responsible  for  the  day-to-day  operation  of  the  laboratories,  and 
the  graduate  students.  These  discussions  centered  on  model  structure  and  user 
considerations.  The  questionnaire  was  used  as  a  basis  for  our  discussions, 
but  the  discussions  were  not  restricted  to  questionnaire  items. 

Model  Considerations 


Adequacy  of  the  model  in  meeting  potential  user  requirements  was  a 
principal  concern.  The  underlying  assumptions  used  to  develop  each  model  v/ere 
considered  important,  since  these  included  anthropometric  considerations  and 
the  range  of  body  positions  incorporated  within  the  model.  The  methods  of 
extrapolating  to  other  body  positions  were  also  of  primary  concern,  as  was  the 
ease  of  adapting  the  models  to  predict  body  and  limb  movement,  restricted 
workspace  considerations,  and  repetition  and  fatigue  effects. 

User  Considerations 


The  most  critical  user  considerations  focused  on  the  current  operability 
of  the  model  and  the  ease  with  which  new  body  positions  could  be  added. 
Additionally,  minimal  input  requirements,  such  as  the  weight  of  the  object  to 
be  lifted,  carried,  pushed,  or  pulled,  and  the  initial  and  terminal  position, 
seemed  desirable.  The  type  of  output  and  time  required  to  obtain  it  were  also 
important  considerations,  as  was  the  capability  of  the  models  to  deal  with 
body  position  or  workplace  restrictions.  Finally,  the  ease  of  making  the 
model  compatible  with  a  computer  aided  design  (CAD)  system  in  the  future  was 
considered. 


Conclusions  and  Recommendations 


We  concluded  that  the  technology  is  available  at  this  time  to  develop  a 
strength  model  that  can  be  used  in  the  design  phase  of  system  development  or 
to  answer  basic  strength  questions  for  existing  systems.  This  conclusion  was 
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based  on  the  responses  to  the  questionnaires,  our  examination  of  the 
facilities,  and  the  discussions  with  the  individuals  at  those  facilities.  The 
model /design  aid  could  be  made  to  handle  questions  related  to  the  full  range 
of  male  and  female  populations  in  various  work  positions.  Confined  spaces, 
clothing  encumberances  and  environmental  effects  could  also  be  accounted  for 
by  modifying  current  biodynamic  strength  models.  This  type  of  system  could 
give  rapid  and  accurate  inputs  to  the  designers  when  the  inputs  are  really 
required,  i.e.,  early  in  the  design  process;  and  it  could  reduce  the  need  for 
redesign. 

As  other  investigators  have  noted,  comparative  evaluation  of  the  thret 
models  is  difficult  because  the  basic  assumptions  underlying  the  models 
dissimilar  (Garg  &  Ayoub,  1980;  Ayoub,  Mital,  Asfour,  &  Bethea,  1980.  A_,  r.^r 
Mital,  Bakken,  Asfour,  &  Bethea,  1980).  There  are  also  differences  ^  no*. 
models  are  constructed,  and  procedures  and  personnel  used  to  develop 
reliability  and  validity  estimates  which  preclude  straightforward 
comparisons.  All  models  reflect  the  competence  of  the  developers;  ana 
selection  could  not  be  based  on  the  central  assumptions  of  the  models  s-n^ 
all  have  been  validated  within  constraints  noted  above.  There  seem  to  Dt  n, 
inherent  reasons  why  one  methodology,  psychophysical  or  biomechanical,  shou' ; 
prove  superior  to  the  other. 

User  interface  requirements  appear  as  a  more  useful  class  of  criteria  for 
selecting  a  biodynamic  strength  model.  The  selection  of  a  model  that  is 
sensitive  to  user  needs  would  provide  a  tool  immediately  useful  to  the  U.S. 
Navy  and  MDC.  The  University  of  Michigan  model  is  the  most  user-responsive 
model.  It  requires  limited  input  information  and  minimal  computer  skills  of 
the  user,  and  produces  graphic  depictions  of  the  final  posture  as  one  type  of 
output.  Therefore,  a  portion  of  the  software  required  for  integration  of  the 
model  into  a  CAD  system  already  exists.  MDC  could  adapt  this  model  to 
accommodate  unique  body  positions  and  tasks,  and  to  add  repetitive  tasks  and 
fatigue  effects  to  fulfill  the  user  requirements  that  MDC  has  identified. 
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Comparing  Engineering  Psychology  and  Industrial  Engineering 
Approaches  to  Logistics  Problems 


Robin  L.  Keesee 


University  of  Louisville 
Louisville,  Kentucky 


The  Issues 


The  Question 


When  the  opportunity  to  speak  at  this  meeting  was  first  presented,  the 
suggestion  was  made  that  the  paper  could  be  on  methodological  concerns  rather 
than  a  reporting  of  experimental  results.  This  was  met  without  great 
enthusiasm  because  there  had  been  no  notable  "visions"  to  be  shared.  But, 
after  that  exchange  and  in  the  midst  of  some  contracted  management  engineering 
studies,  there  was  the  realization  that  although  the  management  engineering 
work  was  being  undertaken  within  an  organization  that  was  purely  industrial 
engineering,  the  strictly  industrial  engineering  methods  were  inadequate  and 
the  methods  used  were  actually  a  hybrid  of  human  factors  and  industrial 
engineering. 

Indeed,  after  participating  in  several  efforts  that  were  similar  to 
logistics  studies,  none  could  be  recalled  that  had  been  performed  entirely 
with  the  methods  that  were  to  be  found  in  either  a  human  factors,  or  an 
industrial  engineering  curriculum.  Perhaps  the  reason  why  defense  logistics 
has  not  been  properly  served  by  human  factors  is  because  the  traditional  human 
factors  methodologies  are  not  appropriate  to  logistics  problems.  This 
possibility  led  to  the  decision  to  deliberately  consider  methods  for 
man-machine  or  human  factors  studies  in  logistics. 

Human  factors  and  industrial  engineering  have  been  around  for  thirty  years 
or  more  as  recognized  professional  disciplines.  If  this  is  the  case,  why  is 
defense  logistics  in  such  a  poor  state  from  a  human  factors  point  of  view? 

Are  there  some  basic  characteristics  of  the  disciplines  that  could  explain 
this  state?  While  this  issue  could  possibly  be  reduced  to  a  formal  hypothesis 
and  tested  empirically,  this  paper  will  depend  on  some  limited  literature 
references  and  on  personal  introspection  and,  very  likely,  rationalization  by 
the  author. 
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As  a  vehicle  for  the  discussion,  engineering  psychology  and  industrial 
engineering  will  be  used  as  extremes  whose  approaches  to  logistics  problems 
will  be  compared.  Hopefully,  the  contrast  in  their  approaches  will  be 
enlightening. 

Note  that  logistics  will  be  used  here  as  "the  procurement,  maintenance, 
and  transportation  of  military  material,  facilities,  and  personnel"  (Websters, 
1965,  pp.  497).  Maintenance  and,  so,  maintainability,  is  explicitly  part  of 
this  common  definition.  The  discussion  will  refocus  on  maintainability  in  the 
conclusions. 

Basis  for  the  Question 

There  are  several  reasons  for  approaching  the  issue  of  human  factors 
problems  in  logistics  in  this  manner.  First,  human  factors  work  is  usually  at 
the  interface  between  man  and  machine  and  requires  knowledge  in  both 
directions.  That  is,  knowledge  is  required  of  both  engineering  and 
psychology.  Evidence  of  this  union  of  two  traditional  disciplines  is  seen  in 
the  membership  of  the  Human  Factors  Society  (HFS)  where  55  percent  of  the 
members  are  psychologists  and  16  percent  are  engineers  (Knowles,  1981).  No 
other  academic  speciality  exceeds  10  percent  of  the  membership.  Within 
psychology,  more  members  report  their  academic  training  as  having  been 
"general  psychology"  than  other  specialities  (Table  1).  Note  in  the  table 
that  the  majority  of  HFS  members  trained  in  psychology  have  doctorates. 

For  the  contrast  with  engineering,  engineering  psychology  was  chosen  as 
the  psychology  specialty  for  two  reasons:  First,  it  is  the  name  used  by 
psychologists  in  the  early  American  writing  on  human  factors  as  a 
self-descriptor.  And  second,  if  we  admit  that  human  factors  is  a  hybrid  of 
engineering  and  psychology  and  not  purely  either,  then  engineering  psychology 
is  the  specialty  within  psychology  that  best  represents  the  knowledge  of  human 
performance  relevant  to  the  design  of  the  man-machine  interface.  Here, 
engineering  psychology  is  not  synonomous  with  human  factors. 

To  match  engineering  psychology,  it  appears  that  industrial  engineering  is 
the  appropriate  one  of  the  major  engineering  disciplines  to  represent 
engineers  in  this  comparison.  From  the  time  of  Frederick  W.  Taylor  and  Frank 
and  Lillian  Gilbreth,  Industrial  engineers  have  been  concerned  with  material 
movements  and  work  methods  (Blair  &  Whitston,  1971,  pp.  11).  One  could  say 
that  the  Industrial  engineer  is  concerned  with  the  engineering  of  the  civilian 
equivalents  of  logistics  systems.  Industrial  engineering  is  the  academic 
background  of  a  plurality  of  the  members  of  the  HFS  identified  as  engineers 
(Table  1),  but  in  contrast  with  psychology,  most  of  the  engineers  who  are  HFS 
members  have  the  masters  as  the  final  degree,  again  a  plurality. 

A  final  preliminary  observation  is  In  order  whose  relevance  will  be  shown 
In  the  conclusions.  Automation  makes  more  control  functions  possible  and 
allows  control  with  greater  accuracy  for  the  same  labor,  and  captial 
Investments  and  permits  the  networking  of  previously  independent  entitles. 

This  automation  makes  systems  more  complex.  As  complex  systems  are  installed 
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Table  1 


Academic  Training  of  Psychology  and  Engineering  Members 
of  the  Human  Factors  Society  {Knowles,  1981,  p.  145) 


By  Speciality 


By  Degree 


Psychol ogy 

General 

25.7 

% 

Bachel or 

Experimental 

18.3 

% 

Master 

Industrial 

Engineering 

Other 

4.2 

2.7 

3.7 

% 

% 

% 

Doctor 

Engineering 

Industrial 

54.6 

5.9 

T 

% 

Bachelor 

Electrical 

2.1 

% 

Master 

Mechanical 

% 

Doctor 

and  become  operational,  they  are  likely  to  partially  fail  but  continue  to 
operate  at  a  reduced  performance  level.  Because  of  the  high  investment  in 
these  systems,  project  managers  are  increasingly  likely  to  attempt  to  improve 
existing  systems  rather  than  undertake  design  of  a  new  system. 

Again,  the  objective  of  this  paper  is  to  compare  engineering  psychology 
and  industrial  engineering  approaches  to  logistics  system  problems  as  a  way  to 
identify  the  causes  of  human  factors  problems  in  current  systems  that  may  be 
organic  to  the  human  factors  discipline.  The  general  form  of  each  approach, 
the  academic  foundations  of  each  approach,  and  the  relative  strengths  and 
weaknesses  of  each  will  be  considered  in  turn. 


Engineering  Psychol oc 


The  Stereotypical 


It  is  believed  fair  to  say  that  the  usual  approach  of  engineering 
psychology  to  logistics  problems  would  consist  of  the  following  steps: 

1.  Introduction  to  (logistics)  application  area. 

2.  Identify  (and  accept)  likely  personnel  subsystems  problems. 

3.  Propose  (and  accept)  candidate  solutions. 

4.  Design  and  conduct  solution  evaluation  experiments. 

5.  Present  concluding  recommendations. 

The  initial  step  is  in  agreement  with  Davis  and  Behan  (1962,  pp.  490)  who 
are  the  only  authors  identified  who  recommend,  even  implicitly,  that 
engineering  psychologists  may  be  required  to  study  large,  complex  systems 
during  operation.  Their  approach  is  experimental  and  they  provide  guidance  on 
the  conduct  of  an  experimental  program  involving  large  scale  systems  although 
their  guidance  does  not  explicitly  mention  the  steps  just  given. 
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During  the  initial  study  phase  where  some  system  familiarity  is  gained, 
the  Investigating  engineering  psychologist  is  likely  to  receive,  either 
spontaneously  or  at  his  query,  suggestions  from  systems  managers  or  operators 
that  training,  selection,  work  station  design,  etc..  Is  a  problem.  Once  the 
personnel  subsystem  problem  is  identified,  the  engineering  psychologist 
develops  alternatives  that  may  solve  the  problem,  and  proposes  experimental 
evaluation  of  the  proposed  solutions.  Based  on  an  inferential  statistics  test 
of  the  results,  the  engineering  psychologist  makes  recommendations  about 
whether  the  solution  is  adequate  or  whether  more  development  is  needed.  If  it 
is  agreed  that  this  Is  the  stereotypical  approach,  why  do  stereotypical 
engineering  psychologists  behave  in  this  manner?  Looking  at  the  foundations 
for  this  approach  may  help  answer  this  question. 

Foundation  of  the  Approach 

Definitions.  To  begin  at  the  most  simple  level,  consider  the  definition 
of  engineering psychology.  A  common  definition  of  the  parent,  psychology,  is 
that  "psychology  is  the  science  of  behavior"  (Gagne  &  Fleishman,  1959,  p.  1). 
McCormick  (1970,  pp.  3)  brings  the  definition  closer  by  saying  that  human 
factors  engineering  is  the  "process  of  designing  for  human  use."  In 
attempting  to  partition  the  psychologies  relevant  to  systems,  de  Greene  (1970, 
pp.  44)  states  that  'Engineering  psychology. . .Is  mainly  concerned  with  the 
design  of  equipment,  facilities,  and  environments  to  match  the  capabilities 
and  limitations  of  people."  To  combine  these  three,  engineering  psychology 
will  be  defined  here  as  the  science  of  human  behavior  relevant  to  the  design 
of  the  man-system  Interface  where  systems  may  be  any  combination  of  hardware, 
software,  and  personnel. 

The  clear  understanding  in  these  definitions  is  that  the  expected  context 
of  engineering  psychology  applications  is  the  design  of  new  systems. 

Education.  One  way  to  look  at  the  tools  of  inquiry  and  analysis  that  are 
the  property  of  an  engineering  psychologist  is  to  see  what  tools  are  provided 
through  his  education.  If  the  assumption  is  made  that  those  practicing 
engineering  psychology  are  represented  in  the  HFS  membership,  it  will  be 
recalled  from  Table  1  that  the  majority  of  psychologists  have  the  doctorate. 
That  Is  the  education  program  that  will  be  considered. 

Doctoral  programs  in  engineering  psychology  typically  consist  of 
significant  course  work  in  three  or  four  primary  areas.  The  first  of  these 
might  be  called  engineering  psychology  Itself:  courses  giving  the  knowledge 
of  hum*)  behavior  relevant  to  the  design  of  hardware.  These  would  include 
sensory,  perceptual,  and  cognitive  processes,  and  perhaps  work  or  exercise 
physiology  and  engineering  anthropometry.  The  second  major  area  would  be 
Inferential  statistics  and  would  include  hypothesis  testing,  analysis  of 
variance,  nonparametric  statistics,  and  very  likely  one  or  more  advanced 
topics  such  as  multivariate  analysis  or  response  surface  methodology. 

The  final  two  areas  of  commonplace  emphasis  are  less  standard  in 
structure.  Most  programs  have  one  or  more  courses  on  methods  of  psychological 
research,  experimental  methods,  psychological  scaling,  psychophysics,  etc. 
Last,  In  programs  that  vary  significantly  from  experimental  psychology  by 
specializing  in  engineering  psychology,  there  will  probably  be  a  few  courses 


on  human  factors  Involvement  in  design  of  complex  man-machine  systems  covering 
task  analysis,  function  allocation,  qualitative  and  quantitative  personnel 
requirements  Information  (QQPRI),  and  maybe  overviews  of  selection  and 
training  within  the  context  of  the  research,  development,  and  initial  fielding 
phases  of  the  life-cycle  of  the  hardware  system. 

The  thrust  of  the  undergraduate  and  early  graduate  psychology  education  is 
on  explanation  of  human  behavior  frequently  using  qualitative  models  with  much 
of  the  later  instruction  keyed  to  classical  experimentation.  The  extension  of 
this  thrust  in  later  graduate  education  is  on  procedural  and  analytical  tools 
necessary  for  experimentation. 

Observations  on  Performance 

In  the  three  full  decades  that  engineering  psychologists  have  been  part  of 
defense  RAD,  there  has  been  a  steady  stream  of  new  systems.  In  these 
developments,  engineering  psychologists  have  made  recommendations  to  improve 
system  performance  or  to  avoid  poor  performance.  Often,  no  objective  evidence 
was  obtained  to  validate  the  worth  of  the  recommendation  in  the  end  product 
but  survival  of  the  discipline  within  the  defense  community  suggests  a  net 
positive  benefit. 

In  defense  logistics,  however,  there  have  been  no,  or  few  newly  developed 
systems  on  which  engineering  psychology  could  be  employed  or  that  could  be 
used  to  fund  advances  in  the  state  of  the  art.  Indeed,  defense  logistics  is 
composed  of  operating,  not  developmental  systems— systems  that  are  working 
even  though  perhaps  not  as  desired  or  not  at  full  potential. 

The  advantage  the  engineering  psychologist  has  in  working  on  logistics 
systems  over  weapon  systems  is  that  the  logistics  systems  work  in  peacetime, 
making  observation  and  data  collection  during  operation  feasible  while  combat 
weapons  systems  do  not,  although  the  logistics  systems'  peacetime  volume  and 
environment  is  probably  dissimilar  to  its  wartime  situation.  The  engineering 
psychologist's  chief  disadvantage  in  addressing  logistics  is  that  there  are 
few  "clean  sheet  of  paper"  developments  on  which  his  design  oriented  tools 
could  be  applied. 

If  the  engineering  psychologist  took  the  view  of  a  unit  mechanic  rather 
than  a  new  hardware  development  project  manager's  view  ( 1 .e. ,  M-l  tank  or 
F/A-18  aircraft),  then  task  analysis,  a  design  oriented  engineering  psychology 
tool,  could  be  usefully  applied  to  a  logistics  system. 

Overall,  engineering  psychology  has  apparently  performed  well  in  design  of 
new  systems  but  has  been  less  effective  in  logistics. 

Case  Studies  from  Engineering  Psychology 

In  the  author's  experience,  there  are  three  significant  studies  that 
involved  efforts  to  improve  systems  that  were  complex  and  that  were 
operating.  Two  of  these  were  in  settings  close  to  engineering  psychology  and 
one  was  in  an  industrial  engineering  environment.  The  methods  used  in  these 
studies,  and  their  success,  speak  to  the  current  issues. 


Case  study --NYCMTA.  Several  Incidents  and  accidents  in  1970  had 
apparently  been  poorly  handled  by  the  senior  operations  supervisors  of  the  New 
York  City  subway,  leading  to  some  traumatic  experiences.  In  late  1970,  the 
executive  management  of  the  subway  division  of  the  New  York  City  Metropolitan 
Transit  Authority  (NYCMTA)  invited  a  group  of  faculty  from  Virginia 
Polytechnic  Institute  Initially  to  develop  a  remedial  training  program  for 
these  senior  operations  supervisors,  the  Desk  Trainmasters.  In  an  Initial 
consulting  effort,  the  faculty  group,  all  but  one  of  whom  were  psychologists 
and  none  of  whom  were  Industrial  engineers,  became  familiar  with  the 
operations  environment,  collected  and  analyzed  performance  data,  and  provided 
a  comprehensive  list  of  recommendations  not  limited  to  training  (Snyder, 
Wlerwllle,  Sgro,  A  Torgersen,  1972).  There  were  several  significant  parts  to 
the  data  collection  and  analysis  program.  A  task  analysis  was  conducted  for 
the  Desk  Trainmaster  position  using  communications  recordings  from  past 
emergency  incidents  as  the  scenarios.  A  human  engineering  review  of  the 
communications  and  Information  storage  and  retrieval  equipment  was  conducted. 
Trainmasters  were  Interviewed  for  their  opinions  on  training  requirements. 
Information  necessary  In  emergencies,  and  strategies  for  resolving 
emergencies.  Finally,  a  time  study  through  the  a.m.  and  p.m.  rush  hours  was 
conducted  to  determine  task  loading  in  nonemergency  situations,  time  and  error 
performance  in  communications  tasks,  and  the  extent  of  nonelectrical 
communications  within  the  command  center. 

To  be  doctrinaire,  one  could  classify  task  analysis,  interviews,  and  human 
engineering  reviews  as  being  from  engineering  psychology,  and  the  time  study 
as  being  from  industrial  engineering. 

More  interesting,  approximately  ten  man-weeks  of  time  from  professionals 
(18  if  time  of  BS-Engineerlng  graduate  students  were  included)  were  devoted  to 
studying  the  situation  to  identify  the  human  behavioral  problems  and  to 
estimate  their  relative  magnitude.  Once  the  specific  problems  were 
identified,  approximately  four  professional  man-weeks  were  spent  in  the 
solution  implementation  phase  although  more  than  24  man-weeks  of  technician 
resources  were  committed  In  this  phase.  Given  a  total  manpower  expenditure  of 
46  man-weeks,  22  percent  of  these  were  professional  (39  percent  if  the  8 
man-weeks  of  BS-Engineers  are  counted)  and  were  spent  identifying  the  specific 
problems.  Of  the  total  professional  expenditure  of  14  man-weeks  (or  22 
man-weeks),  71  percent  (or  82  percent)  were  devoted  to  figuring  out  what  the 
problem  was  with  the  remainder  devoted  to  supervising  solution  Implementation 
which  was  the  longer  phase  from  a  calendar  view. 

Case  study— Army  repair  parts  supply.  A  study  less  equivocally  in 
logistics  was  one  conducted  by  the  U.S.  Army  Human  Engineering  Laboratory 
(HEL)  that  sought  to  resolve  human  performance  problems  In  the  retail  repair 
parts  supply  system  (Keesee,  Camden,  Powers,  Kilduff,  Hill,  Gombash  A  Sarll, 
1980).  Being  within  the  HEL,  the  study  was  In  an  engineering  psychology 
environment.  Data  were  collected  using  three  general  methods.  Time  studies 
were  made  of  repair  parts  clerks  to  learn  job  content,  task  load,  and  task 
time  and  error  performance,  and  to  gain  detailed  familiarity  with  the  jobs  to 
help  uncover  possible  work  methods  Improvements.  Structured  Interviews  with 
the  clerks,  their  supervisors,  and  technical  and  operations  managers  were  held 
to  gain  their  opinions  of  sources  of  problems  and  the  perceived  distribution 


of  time  among  tasks.  Finally,  a  simulation  model  was  developed  of  the  repair 
parts  request  paper  work  and  part  flow  from  the  customer  units  through  the 
Division  and  to  higher  echelons.  This  model  was  Intended  to  be  a  good 
framework  for  the  collected  data  and  a  guide  to  the  limits  and  bottlenecks  on 
system  performance. 

Of  the  methods  used,  the  structured  Interview  and  associated  content-type 
analysis  were  from  engineering  psychology  while  the  time  study  and  the 
simulation  were  from  Industrial  engineering.  As  In  the  first  case  study,  a 
look  at  the  resource  phasing  may  also  be  Instructive.  Of  the  approximately 
108  man-months,  78  (72  percent)  were  In  the  study  phase  where  study  team 
members  became  familiar  with  the  system,  devised  and  Implemented  a  data 
collection  and  analysis  plan,  and  finally  Identified  specific  problems  in 
human  behavior  and  performance  terms.  The  remaining  28  percent  of  the  effort 
was  In  developing  the  recommended  redesign  of  the  system. 

Summary 

If  the  admittedly  limited  evidence  Is  accepted,  it  has  been  observed  to 
this  point  that  engineering  psychology  has  Its  roots  deeply  In  experimentation 
which  Is  comparative  by  nature,  that  most  of  the  professional  time  in  a  study 
of  a  logistics  system  is  spent  on  determining  the  narrow  Identity  of  the 
problem  and  much  less  time  Is  spent  on  designing  solutions,  and  that 
Investigation  of  systems  In  operation  may  employ  data  collection  methods  not 
all  of  which  are  classically  engineering  psychology. 

Industrial  Engineering 

The  Stereotypical  Approach 

Faced  with  a  military  logistics  assignment,  the  Industrial  engineer  will 
probably  go  through  the  following  steps: 

1.  Introduction  to  (logistics)  application  areas. 

2.  Identify  likely  areas  of  Inefficient  operations. 

3.  Collect  and  analyze  time  performance  and  cost  data. 

4.  Develop  and  evaluate  alternative  solutions. 

5.  Recommend  and  justify  solution. 

In  the  first  step,  the  Industrial  engineer  would  become  familiar  with  the 
operations  to  be  studied  helped  somewhat  by  the  similarity  to  civilian 
Industry.  Instead  of  trying  to  stimulate  discussion  with  supervisors  and 
workers  about  methods  of  selecting  and  training  the  operators  as  might  the 
engineering  psychologists,  the  Industrial  engineer  Is  likely  to  Inquire  about 
the  lengths  of  queues,  workload  fluctuations,  work  planning  and  scheduling, 
etc. 

After  gaining  familiarity  with  the  overall  operation,  the  Industrial 
engineer  will  probably  choose  one  or  two  situations  that  appear  to  be 
Inefficient  because  of  being  labor  Intensive  or  duplicative,  having  extensive 
static  Inventory,  having  Idle  workers  or  equipment,  or  having  lengthy  waiting 
times  for  customers.  Once  a  problem  area  has  been  Identified,  time  and  cost 
data  will  be  collected  and  analyzed.  Alternatives  will  then  be  developed  and 


cost  and  time  performance  for  these  will  be  estimated.  A  solution  will  then 
be  recommended  with  quantitative  evidence  that  the  recommended  solution  Is  the 
best  of  the  alternatives  and  significantly  better  than  the  current  methods. 

Foundation  of  the  Approach 

Definitions.  With  the  first  separate  academic  industrial  engineering 
departments  formed  In  1908  at  Penn  State  and  Syracuse  (Turner,  Mize,  A  Case, 
1978,  pp.  22),  Industrial  engineering  is  a  little  older  than  engineering 
psychology.  In  1955,  the  American  Institute  of  Industrial  Engineers  agreed 
upon  a  definition: 

Industrial  Engineering  Is  concerned  with  the  design.  Improvement,  and 
Installation  of  Integrated  systems  of  men,  materials,  and  equipment.  It 
draws  upon  specialized  knowledge  and  skill  In  the  mathematical,  physical, 
and  social  sciences  together  with  the  principles  and  methods  of 
engineering  analysis  and  design  to  specify,  predict,  and  evaluate  the 
results  to  be  obtained  from  such  systems  (Turner,  et  al.,  1978,  pp.  21). 

Mote  that  Industrial  engineering  Is  the  only  major  engineering  discipline 
to  explicitly  Include  people  as  a  component  of  the  design  process.  Also,  at 
least  the  definition  refers  to  systems  improvement  as  well  as  design. 

Education.  Since  the  HFS  directory  Indicated  that  more  industrial 
engineering  members  had  masters  degrees  than  had  doctorates,  that  will  be  the 
degree  program  considered. 

Besides  the  basic  mathematics  and  science  requirements  of  the  early 
undergraduate  processes,  the  Industrial  engineer  will  have  required  courses  in 
the  major  discipline  In  four  or  five  categories  with  a  few  possible 
electives.  Most  fundamental  would  be  a  course  In  methods  engineering  covering 
time  studies,  activity  charts,  work  sampling,  synthetic  motion  time  systems, 
etc.  Similarly  fundamental  would  be  courses  in  accounting  and  economic 
decision  making.  At  the  Intermediate  level  would  be  human  behavior  In 
organizations,  a  survey  of  engineering  psychology,  and  possibly  work 
physiology.  Inferential  statistics  would  be  taught  in  two  or  more  courses 
that  would  consider  hypothesis  testing,  analysis  of  variance,  regression,  and 
quality  control.  Coursework  In  operations  research  would  cover  queueing, 
facilities  layout,  scheduling,  Inventory  and  other  process  models, 
optimization,  mathematical  statistics,  and  digital  simulation. 

At  the  masters  level,  additional  courses  In  the  process  models, 
simulation,  mathematical  statistics,  and  the  mathematics  underlying  operations 
research  would  be  taken  If  operations  research  were  the  major.  Manual 
control,  safety,  display  design,  human  factors  In  design  of  complex  systems, 
and  more  Inferential  statistics  courses  would  be  taken  by  human  factors  majors. 

The  thrust  of  the  undergraduate  and  early  graduate  industrial  engineering 
education  Is  primarily  In  quantitative  solutions  to  problems  especially 
problems  Involving  time,  costs  and  stochastic  variables.  A  secondary  emphasis 
Is  In  management  of  technical  or  manufacturing  operations.  There  Is  little 
emphasis  on  seeking  understanding  of  underlying  processes  unless  a 
quantitative  relationship  among  events  Is  expected. 


Observations  on  Performance 


The  Accrediting  Board  for  Engineering  and  Technology  (ABET)  insures  that 
academic  programs  maintain  a  balance  in  course  content  between  industry's 
typically  immediate  needs  and  academicians'  perceptions  of  what  will  be  needed 
by  the  engineer  practicing  over  the  next  ten  to  thirty  years.  The  continued 
demand  for  graduates  suggests  that  industrial  engineers  are  being  adequately 
trained  to  meet  industry's  needs  for  quantitative  management  staff. 

In  defense  logistics,  the  BS/MS  general  industrial  engineer  is  likely  to 
do  well  at  improving  worker  task  performance  including  maintenance  tasks.  The 
industrial  engineer's  training  would  directly  apply  to  synthesizing  and 
simplifying  maintenance  and  operating  tasks  where  hardware  does  not  yet  exist 
and  to  perform  time  and  motion  studies  where  hardware  does  exist.  This  latter 
work  would  be  of  the  nature  of  that  reported  by  Theisen  and  Hsu  (1982)  as 
secondary  to  their  research  study.  The  BS/MS  IE  often  spends  the  first  year 
or  two  out  of  school  in  methods  of  work  simplification.  Such  an  engineer 
would  consider  methods  improvement  and  simplification  of  maintenance  tasks  to 
be  a  routine  and  unremarkable  application  of  his  skills. 

However,  the  IE,  without  a  specialization  in  human  factors  at  the  MS 
level,  is  unlikely  to  have  a  highly  developed  global  view  of  the  systems.  That 
is,  without  one  or  more  courses  concentrating  on  interactions  within  elements 
of  the  personnel  subsystem,  the  BS/MS  IE  is  prone  to  concentrate  on  improving 
the  efficiency  of  system  or  process  subsystems,  treating  the  humans  as  an 
incidental  randomly  behaving  component. 

A  Case  Study  from  Industrial  Engineering 

Recently  the  U.S.  Naval  Ordnance  Station-Louisville  asked  the  Department 
of  Industrial  Engineering  of  the  University  of  Louisville  for  assistance  in 
conducting  management  engineering  studies  of  a  collection  of  service  functions 
such  as  custodial  service,  administrative  telephone  operations,  vehicle 
operations  and  maintenance,  etc.  The  objective  of  the  studies  was  to  improve 
labor  and  material  efficiency  in  each  function. 

The  methods  of  study  selected  by  the  faculty  group  leading  the  effort  were 
structured  interviews  to  gain  worker  and  supervisor  opinions  about  the  current 
nature  of  the  v/ork  and  possible  improvements,  review  of  accounting 
information,  work  sampling  in  functions  where  workers  had  fixed  v/ork  stations, 
and  a  modified  time  study  in  all  areas  that  was  similar  to  what  Mundel  (1978, 
pp.  94)  called  work  activity  analysis.  This  modified  time  study  sought  to 
determine  the  job  content,  to  make  some  estimates  of  the  accuracy  of 
accounting  information,  and  to  force  the  analysts  to  gain  great  familiarity 
with  the  job  in  order  to  identify  methods  improvements. 

Obviously,  the  time  study,  work  sampling,  and  use  of  accounting  data  are 
classical  industrial  engineering  tools  while  the  structured  interview  is  more 
in  the  engineering  psychology  direction.  Looking  at  phasing  of  resources,  it 
appears  that  seventeen  man-months  of  professional  time  will  be  expended  in 
Identifying  areas  of  inefficiency  or  problems  with  perhaps  eight  man-months 
spent  in  developing  specific,  implementable  recommendations. 


Surma  ry 

From  the  review  of  the  industrial  engineer's  education  and  one  limited 
example,  it  appears  that  this  discipline  is  trained  to  solve  defined  problems 
Involving  randomly  varying  costs,  operations  volumes,  and  times 
quantitatively.  More  time  will  be  spent  by  the  industrial  engineer  in 
defining  the  problems  than  in  developing  a  solution. 

The  Comparison 

Results 

Given  training  in  the  human  factors  methods  appropriate  to  design  of  a 
complex  man-machine  system,  function  allocation,  task  analysis,  QQPRI,  etc., 
the  engineering  psychologist  will  have  a  whole  system  perspective.  This  view 
may,  however,  be  focused  on  the  personnel  subsystem  and  only  dimly  include 
overall  logistic  systems  performance.  The  industrial  engineer  will  be  more 
concerned  with  improving  one  subsystem  at  a  time  but  would  use  the  impact  of 
the  recommended  solution  on  the  total  system  cost,  time,  or  other  quantitative 
performance  measure  to  justify  acceptance  of  the  recommendation.  The 
industrial  engineer  would  be  able  to  make  improvements  in  the  physical  work 
place  but  would  tend  to  not  recognize  the  human  information  processing 
limitations  that  the  engineering  psychologist  would  note.  It  would  appear 
that  neither  engineering  psychology  nor  industrial  engineering  is  directly 
appropriate  to  logistics  studies  although  both  are  useful  training  for  the 
field.  Neither  Is  complete. 

Discussion 

Basic  approaches  of  engineering  and  psychology  vs.  logistics  studies.  In 
the  comparison  above,  stereotypical  approaches  to  logistics  problems  were 
hypothesized  for  engineering  psychology  and  industrial  engineering.  Consider 
their  approaches  to  their  natural  work.  Virtually  every  introductory  or 
intermediate  engineering  text  includes  some  version  of  the  engineering  problem 
solving  process.  Krick  (1962,  pp.  15-20)  lists  several.  A  typical  version  of 
the  engineering  approach  to  problems  is: 


1.  Define  problem. 

2.  Define  constraints. 

3.  Develop  alternatives. 

4.  Evaluate  alternatives. 

5.  Decide  and  Implement. 

The  formal  research  process  in  engineering  or  experimental  psychology 
could  be  represented  by  the  following  steps: 

1.  Define  hypothesis. 

2.  Design  and  conduct  an  experiment. 

3.  Analyze  results. 

4.  Generalize  results  from  experiment  to  at  least  the  hypothesis. 

Both  of  these  approaches  begin  with  what  is,  in  effect,  a  statement  of  the 
objective.  In  light  of  the  three  case  studies,  it  seems  that  stating  the 


initial  step  so  blithely  gives  little  credit  to  the  difficulty  of  the  task. 

In  each  of  the  three,  the  vast  majority  of  the  work  effort  was  in  defining 
problems  that  were  solvable.  Logically,  the  steps  that  are  undertaken  in 
logistics  studies  should  reflect  the  necessary  sequence  of  progress  in  the 
work.  A  reasonable  sequence  might  be  the  following: 

1.  Diagnosis. 

2.  Define  limiting  factors. 

3.  Select  problems. 

4.  Solve  with  appropriate  technical  discipline. 

There  will  be,  in  the  future,  an  increasing  rate  of  requests  for  human 
factors  assistance  to  help  improve  logistic  systems  as  well  as  to  help  improve 
operational  performance  of  other  complex  systems  that  have  already  been 
designed  and  fielded.  The  first  step  in  these  efforts  is  properly  entitled 
diagnosis  and  represents  the  gathering  of  valid  performance  data,  and  the 
analysis  of  these  data  for  comparison  with  norms,  standards,  specifications, 
and  expectations  to  arrive  at  the  precisely  stated  problems  that  are  amenable 
to  solution. 

The  second  step,  one  that  begins  midway  through  the  diagnosis  phase,  is 
the  listing  of  all  of  the  problems  or  circumstances  that  hinder  system 
performance,  the  limiting  factors.  Items  on  this  list  should  then  be 
objectively  evaluated  for  effect  on  system  performance.  Those  that  are  the 
most  significant  and  are  capable  of  being  solved  within  available  resources 
should  be  selected  and  solved  by  whatever  technical  discipline  is  appropriate. 

Methodology  considerations  in  logistics  system  diagnosis.  Several  things 
of  varying  importance  have  been  learned  in  the  experiences  reported  above 
concerning  diagnosis  of  logistics  systems.  First,  a  quantitative  model  of 
systems  operation  and  performance  should  be  developed  beginning  early  in  the 
diagnosis  phase.  This  model  will  serve  as  a  guide  to  determine  what  operating 
data  should  be  gathered  and,  once  the  data  is  in  hand,  the  model  will  serve  as 
a  framework  for  utilization  of  the  data.  Also,  the  model  will  reflect  how 
well  the  diagnosticians  understand  the  system  operation. 

Because  operating  logistics  systems,  and  to  some  extent  other  operating 
complex  systems,  frequently  have  management  Information  systems  (MIS)  that 
collect,  summarize,  and  report  an  abundance  of  statistics,  operational 
performance  data  is  usually  readily  available.  The  diagnostician  should 
accept  all  offers  of  such  data,  and  should  pursue  this  source  as  well  as  the 
often  more  useful  data  hidden  away  in  the  manually  kept  logs  and  registers  of 
Individual  units.  The  pursuit  of  these  existing  data  sources  and  acceptance 
of  all  offers  will  possibly  reduce  the  necessity  to  Independently  collect  data 
or  will  make  the  data  that  is  collected  much  more  relevant.  But  a  caution  is 
in  order  here.  Since  the  performance  statistics  from  MIS  are  very  often  the 
basis  in  part  for  a  career  performance  rating  of  operations  supervisors  and 
managers,  the  MIS  may  be  susceptible  to  manipulation.  While  useful,  the  award 
of  credibility  of  the  whole  of  the  MIS  results  should  be  reserved  until 
validity  of  the  data  can  be  determined. 

The  remaining  comments  on  diagnosis  methodology  are  of  less  importance. 

In  diagnosing  problems  of  a  logistics  system,  opinions  of  the  system  users  or 


operators  should  be  represented.  Without  their  perception  of  participation, 
implementation  of  the  results  will  be  thwarted.  The  individual  responsible 
for  conduct  of  the  logistics  study  should  participate  in  any  data  gathering  or 
collection  to  help  ensure  that  the  limitations  on  generalizing  of  the  data 
results  are  learned  and  respected.  A  last  note  is  that  it  is  unlikely  that 
the  quantitative  model  will  be  completed  within  the  study  schedule  but  will  be 
useful  nonetheless. 

Ramifications  for  professional  education.  If  it  is  intended  that  human 
factors  professionals  be  employed  to  improve  existing,  complex,  operational 
systems,  then  their  education,  either  psychology  or  engineering  based,  should 
reflect  the  extent  to  which  diagnosis  will  consume  their  time  in  professional 
practice.  Only  the  engineering  psychology  subset  of  psychology  terminal 
degree  graduates  need  be  concerned  here.  But  all  industrial  engineering  BS/MS 
graduates  can  expect  to  be  asked  to  attempt  improving  the  whole  of  a  complex 
system,  albeit  a  civilian  system,  at  some  point  in  their  careers.  Yet  no 
engineering  text  has  been  found  that  integrates  the  several  individual  data 
collection  and  analytical  techniques  into  a  strategy  or  set  of  principles  for 
diagnosis.  Alas,  the  only  text  where  use  of  industrial  engineering  techniques 
for  diagnosis  of  whole,  complex  systems  is  even  implied  is  Chapanis  (1959). 
(Methods  engineering  with  its  time  study  and  other  direct  observation 
techniques  is  taught  in  IE  curricula  in  the  context  of  making  improvements 
within  a  subsystem  or  component  of  a  system  or  process.) 

The  strategy  and  techniques  of  diagnosis  and  the  subsequent  steps  listed 
above  should  be  addressed  in  human  factors  graduate  programs  and  in 
undergraduate  industrial  engineering  curricula. 

Conclusions 

Although  it  is  good  that  elements  of  the  defense  human  factors  community 
are  becoming  concerned  with  logistics  problems,  it  would  appear  from  the 
abundance  of  such  problems  that  the  service  provided  logisticians  by  human 
factors  has  been  lacking  in  the  past.  By  comparing  the  approaches  of 
disciplines  on  either  extreme  of  human  factors,  it  has  become  clear  that  human 
factors  as  a  technology  is  oriented  toward  design  of  new  systems  and  not 
toward  improvement  of  systems  that  are  in  place  and  operating.  So,  it  is 
plausible  that  one  reason  that  there  are  significant  human  factors  problems  in 
logistics  systems  is  that  most  logistics  systems  are  products  of  evolution 
rather  than  new  designs  and  the  classical  human  factors  tools  do  not  directly 
apply. 

If  this  conclusion  is  correct,  several  remedies  may  be  undertaken. 

Clearly,  the  basic  and  often  academic  research  that  promises  improvements  in 
selection,  training,  job-aid  design,  and  other  man-system  interface  issues  for 
naintainers  and  other  logistics  personnel  should  continue.  In  structuring 
efforts  to  Improve  a  given  logistics  system,  those  practicing  human  factors 
should  resist  the  temptation  to  begin  comparative  or  experimental  activities 
until  after  completing  a  deliberate  diagnosis  to  identify  and  evaluate  the 
significance  of  all  of  the  personnel  subsystem  problems  within  the  logistics 
system. 


This  counsel  will  apply  equally  to  the  nondefense  and  nonlogistics  within 
defense  human  factors  groups  in  the  future.  As  the  systems  within  their 
purview  increase  in  complexity  from  automation  or  other  sources,  and  their 
managements  seek  improvements  in  lieu  of  costly  new  designs,  the  need  for 
human  factors  diagnosis  methodologies  will  be  similar  to  the  current  needs  in 
defense  logistics. 

Returning  at  last  to  the  conference  topic,  applying  the  strategy  above  to 
maintainability  would  suggest  that  maintenance  within  a  service  should  be 
studied  from  a  human  factors  view  as  a  system.  For  example,  field  maintenance 
of  tanks  might  be  the  study  scope  rather  than  maintenance  of  a  new  hardware 
system  such  as  the  new  M-l  tank.  An  orientation  by  the  human  factors  study 
team  toward  a  particular  hardware  end-item,  an  aircraft  or  a  tank,  will  have  a 
high  potential  for  degenerating  into  a  series  of  specific  fixes.  These  fixes 
will  be  beneficial  but  the  identification  of  required  fixes  is  not  something 
for  which  professionals  in  human  factors  are  typically  well  trained  and  will 
cause  them  to  miss  the  broad  problems  that  are  common  across  hardware  systems 
and  to  which  the  human  factors  community  can  make  major  and  unique 
contributions. 

While  easily  recommended,  it  is  recognized  that  the  current  human  factors 
organizations  within  DOD  are  not  conducive  to  the  application  of  this 
diagnosis  strategy  because  of  their  research  orientation.  Perhaps  their 
organizational  structures  can  be  adapted  to  permit  the  practice  of  human 
factors  application  in  the  suggested  manner  alongside  the  research. 

The  foregoing  observations,  comments,  recommendations,  and  admonitions  may 
seem  to  many  as  gratuitous  coming  as  they  apparently  do  with  no  more  than 
personal  empiricism  as  their  basis.  Any  future  more  credibly  objective 
investigation  of  these  issues  might  formally  survey  the  content  of  academic 
programs  for  relevance  to  human  factors  diagnosis,  and  the  frequency  of 
occurrence  of  various  experimental  and  analytical  methods  in  investigations  of 
complex  operational  systems  reported  in  the  applied  journals.  Unfortunately, 
few  written  reports  reveal  directly  the  relative  manpower  expenditures  between 
narrowly  defining  the  problem  and  solving  it.  Until  the  voids  are  filled, 
perhaps  the  success  of  this  writing  will  be  seen  in  any  subsequent  discussion 
of  methodological  propriety  in  logistics  studies. 
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It  is  a  high  honor,  indeed,  to  be  asked  to  comment  on  what  has  transpired 
here  the  past  two  days.  LT  McBride  is  to  be  commended  for  developing  such  a 
well -integrated  technical  program  in  this  significant  area.  He  has  taken  a 
systems  approach  to  this  important  problem  and  it  is  important  that  each 
contributor  keep  that  in  mind.  Otherwise,  we  will  fall  into  the  “blind  men 
and  the  elephant"  problem,  each  feeling  that  the  parameters  with  which  he 
deals  hold  all  of  the  answers  to  the  problem.  In  reality,  it  is  difficult  to 
think  of  an  area  In  which  selection,  training,  human  engineering, 
environmental  considerations— in  fact,  all  of  the  human  factors 
areas— Interact  more  intimately.  We  might  gently  urge  that,  in  addition  to 
his  emphasis  on  the  establishment  of  quantitative  relationships  between 
maintenance  design  principles  and  cost  and  readiness,  checklists,  modeling  of 
the  maintainer' s  role  in  the  system,  and  the  effects  of  environmental 
variables  (odd  hours,  long  hours,  etc.),  the  interactions  of  these  with 
selection  and  training  variables  be  given  additional  consideration.  (If  the 
Navy  is  like  the  Air  Force,  I  suspect  that  this  might  mean  crossing 
organizational  lines.  Cross  them,  LT  McBride;  please  cross  them.) 

These  past  two  days  we  have  heard  papers  or  comments  from  the  floor 
dealing  with  personnel  issues,  methodological  issues,  diagnostics,  model 
development,  criterion  development,  and  training— quite  a  menu  for  a  two-day 
meeting.  LT  McBride  has  structured  a  program  whose  thrusts  encourage 
interaction  among  the  participants.  I  feel,  also,  that  he  has  organized  and 
fathered  the  first  of  what  will  become  an  annual  event— to  the  benefit  of  the 
other  Departments  of  DOD  and  industry,  as  well  as  to  the  benefit  of  the  U.S. 
Havy.  The  good  things  we  have  witnessed  here  have  implications  for  any 
department  or  organization  that  is  faced  with  maintenance  of  the  products  of 
our  technological  age. 


The  Age  of  Technology 

Regarding  the  era  of  technology,  whose  threshold  we  probably  have  barely 
crossed  (and  thus  no  one  is  in  a  very  good  position  to  estimate  its  eventual 
Impact),  one  wag  observed  that  had  it  not  been  for  Thomas  Alva  Edison,  all  of 
us  would  have  to  resign  ourselves  to  watching  television  by  candlelight'. 


s 


161 


f5- 

IS. 


mm 


m 


Several  well-known  authorities  suggest  that  the  adaptive  processes  of 
mankind  are  being  strained  to  the  limit  by  modern  technology,  even  suggesting 
that  the  rapid  acceleration  of  technology  presages  the  end  of  civilization,  at 
least  as  we  know  it.  But  very  few  want  to  revert  to  the  living  conditions  of 
a  pretechnological  age. 

The  U.S.  Navy  is  a  prime  example  of  a  department  that  takes  full  advantage 
of  the  latest  technological  developments  and,  in  fact,  is  a  leader  among  the 
agencies  that  support  advanced  developments  in  technological  areas.  Like  the 
Air  Force,  where  I  served  for  most  of  n\y  professional  career.  Naval  leaders 
apparently  feel  that  they  must  extract  every  bit  of  capability  from  both  their 
people  and  the  products  of  technology  with  which  those  people  interact.  This 
is  an  easy  position  with  which  to  sympathize  because  it  appears  that  our 
potential  adversaries  are  doing  the  same  thing.  As  human  factors  specialists, 
it  is  our  job  to  apprise  our  leaders  and  systems  planners  and  designers  of 
what  can  be  expected  from  people  in  the  people  x  machines  equations. 

This  symposium  is  evidence  that  the  leadership  of  the  U.S.  Navy  realizes 
that  more  attention  must  be  given  to  the  human  as  a  component  in  the 
maintenance  environment— the  design  of  the  equipment,  the  working 
environment— all  of  the  factors  that  are  being  considered  at  this  meeting. 

The  alternative  is  technological  regression  or  the  acceptance  of  a  maintenance 
burden  that  already  is  reaching  levels  that  preclude  new  developments,  force 
the  Navy  to  buy  two  aircraft  in  order  to  have  one  ready  to  do  its  job,  etc. 

Conditions  qualitatively  similar  to  this  exist  in  non-DOD  pursuits.  A 
comparison  of  the  costs  of  operating  commercial  airlines  in  1970  versus  1980 
may  be  instructive.  (I  am  Indebted  to  United  Airlines  for  the  1980  figures;  I 
forget  v/here  I  got  the  1970  figures.) 


Table  1 

M 

Comparative  Cost  of  Commercial  Airline  Operations:  1970  versus  1980 

i 

1970  (%) 

1980  (%) 

Maintenance 

28 

16 

Fuel  and  oil 

28 

52 

5? 

'-V. 

Flight  crew  salaries 

17 

20 

‘•V 

SB 

>5* 

Depreciation  A  obsolescence 

19 

9 

Miscellaneous 

_8 

_3 

rCy 

TOTAL 

100 

100 

The  1970  figure  of  28  percent  for  maintenance  is  quite  similar  to  that  for 
Navy  and  Air  Force  aviation.  (Of  course,  one  can  also  conclude  from  this 
table  that  the  way  to  reduce  the  percentage  of  expenditures  for  maintenance  is 
to  raise  fuel  prices'.) 
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It  has  been  estimated  that  in  1970  the  cost  of  automotive  transportaton  in 
the  United  States  was  approximately  160  billion  dollars,  and  of  this  amount, 

22  percent  was  spent  on  maintenance.  It  is  further  estimated  that  40  percent 
of  this  22  percent  (or  approximately  14  billion  dollars)  was  wasted  on 
unnecessary  repairs.  We  waste  more  money  on  our  automobiles  than  the  gross 
income  of  many  countries. 

During  this  same  time  period,  American  industry  spent  approximately  200 
billion  per  year  on  maintenance  and  it  is  estimated  that  approximately 
one-third  of  that  amount  was  wasted.  One  industrial  study  showed  that 
corrective  maintenance  is  three  times  as  expensive  as  preventative 
maintenance.  But  before  you  embark  on  an  industrial  program  in  preventative 
maintenance  make  sure  that  the  production  manager  agrees  to  your  requirements 
for  downtime'. 

In  the  American  home,  the  maintenance  proportion  of  the  total  life-cycle 
cost  of  a  color  television  set  in  1977  was  approximately  35  percent,  while  a 
comparable  figure  for  refrigerators  was  6  percent.  There  is  a  lesson  here: 
the  basic  technology  that  supports  a  refrigerator  is  relatively  simple  and  has 
changed  very  little  over  the  past  generation;  we  have  learned  how  to  design 
refrigerators  that  are  almost  maintenance-free.  The  Navy  probably  could 
achieve  that  dream  of  a  95  percent  in-commission  rate  for  Corsairs  today  but 
how  would  you  like  to  engage  a  MI6-25  in  a  Corsair?  No,  the  Navy  will 
continue  to  take  advantage  of  the  latest  technology  and  it  is  incumbent  upon 
people  such  as  us  to  provide  Naval  authorities  and  those  who  develop  their 
systems  with  information  that  will  enable  them  to  integrate  man  into  this 
complex  picture  in  such  a  way  as  to  maximize  systems  effectiveness.  That  is 
why  this  meeting  is  so  important. 

Finally,  just  as  there  is  considerable  transfer  of  technology  from  the 
Navy  to  industry  and  commerce,  it  is  my  belief  that  what  is  being  learned  In 
this,  and  related  programs,  will  benefit  all  of  those  concerned  with  the 
maintenance  and  other  life-cycle  cost  problems  fomented  by  the  products  of 
modern  technology. 

A  Systems  Interpretation  of  This  Seminar 

I  was  weaned  on  the  so-called  "systems  approach"  to  the  design  and 
development  of  systems.  I  hope  that  you  will  indulge  me  if  I  try  to  interpret 
what  I  saw  and  heard  at  this  conference  in  those  terms. 


Figure  1  discloses  that  people,  hardware,  and  job  environment  interact, 
yielding  performance  that  is  Intended  to  meet  specified  system  goals.  The  key 
term  is  interaction.  It  is  worth  noting  that  these  interactions  take  place  in 
(and  Interact  with  the  components  of)  a  "total  environment."  There  is  no 
doubt  that  what  transpires  in  one's  life  outside  the  immediate  job  environment 
affects  his  performance  on  the  job.  Hartman,  of  the  USAF  School  of  Aviation 
Medicine,  has  shown  that  one  of  the  very  significant  factors  that  determines 
how  well  Military  Air  Command  crew  members  perform  Is  the  attitude  of  their 
family  members  toward  the  disrupting  features  of  their  jobs.  We  undoubtedly 
would  find  it  advantageous  to  examine  this  part  of  the  picture,  but  it  must  be 
done  with  care  since  much  of  it  Is  private  and  Involves  other  members  of  the 
family.  However,  It  Is  foolish  not  to  recognize  that  these  factors  can 
sometimes  be  so  powerful  as  to  literally  overwhelm  the  factors  over  which  we 
have  more  direct  control. 


Figure  1.  An  Interactive  model  of  systems  performance. 


Let  us  examine  a  well-known  variance  model  as  a  way  of  establishing  some 
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That  is,  total  systems  variance  is  equal  to  the  sum  of  the  variance 
attributable  to  people  (P),  hardware  (H),  and  environment  (E)  plus  the  three 
first-order  interactions  and  the  single  second-order  interaction.  The 
conference  is  devoting  special  attention  to: 

2  2  2 

0“  (T  and  (T 
PH,  PE,  PHE. 

Professor  Rouse  called  our  attention  to  the  significant  interrelationship 
between  training  and  human  engineering.  Thus  the  0*p  term  probably  should 
be  broken  down  further  to  reflect  such  components  as  aptitude,  education, 
experience,  attitude,  and  motivation.  The  possibilities  of  obtaining 
significant  higher-order  interactions  in  such  a  situation  are  very  real; 
unfortunately,  we  have  very  little  idea  of  how  to  deal  with  them. 

Also,  we  have  said  very  little  at  this  conference  about  the 
interrelationships  between  reliability  and  maintainability,  yet  all  of  us  are 
aware  that  these  interactions  are  very  real  and  very  important.  I  will  only 
observe  that  one  reliability  study  showed  that  only  approximately  30  percent 
of  equipment  failures  were  due  to  poor  workmanship  while  70  percent  were  due 


to  poor  design  and  "careless  handling."  (I  put  the  quotation  marks  around 
"careless  handling"  because  of  the  Implications  for  design.  Something  that  is 
designed  without  handles,  is  awkward  to  handle  and  will  probably  be 
"carelessly  handled.") 

The  model  helps  me,  at  least,  to  conceptualize  what  we  are  stnving  to 
achieve  here.  It  serves  also  to  remind  us  of  significant  factors  that  we  may 
not  be  considering  as  primary  contributors,  and  certainly  not  as  interactive 
contributors.  In  this  particular  application,  the  variance  model  does  little 
more  than  this,  primarily  because  we  usually  don't  have  a  measure  of  0* 

Error.  This  places  severe  restrictions  on  significance  testing'. 

The  variance  model  theoretically  could  handle  such  detailed  structures  as 
Swain's  "Performance  Shaping  Factors"  (Swain  &  Guttman,  1980)  but  it  would 
become  so  cumbersome  as  to  be  useless.  However,  an  examination  of  the  Swain 
table  (Table  2)  serves  to  remind  us  how  far  we  have  yet  to  go  if  we  intend 
accurately  to  describe  and  then  predict  human  behavior  in  complex  systems. 

Personnel  Issues 

Dr.  Modrick  reminded  us  that  we  need  a  better  definition  of  the  population 
for  which  we  are  designing— the  maintenance  population.  About  all  that  we 
ever  hear  is  that,  on  the  average,  they  have  two  years  of  high  school  and  read 
at  a  level  somewhere  between  that  of  a  sixth  grader  and  an  eighth  grader.  We 
know,  also,  that  they  have  virtually  no  mathematical  skills,  that  they  are  not 
particularly  good  at  integrating  information  from  various  sources,  and  that 
they  hate  to  post  information  to  forms'. 

About  15  years  ago,  when  I  was  Director  of  the  Human  Engineering  Division 
of  the  Aerospace  Medical  Research  Laboratory,  we  commissioned  Harold  Leuba  to 
look  into  these  matters  a  bit.  A  few  of  his  findings  come  to  mind:  (1) 
Maintenance  men  have  strong  preferences  as  to  the  troubleshooting  techniques 
they  use.  One  should  force  a  specific  technique  on  them  only  if  it  can  be 
shov/n  to  be  of  considerable  advantage.  (2)  Maintenance  men  have  been  known  to 
report  and  remedy  nonexistent  malfunctions  just  to  Improve  their  track 
records.  (3)  Many  maintenance  men  (and  operators)  have  their  own  favorite 
malfunctions.  Leuba  could  reduce  diagnostic  uncertainty  by  over  50  percent 
simply  by  knowing  who  made  the  complaint. 

The  operators,  of  course,  are  a  party  to  the  current  unsatisfactory  state 
of  affairs.  Some  tend  to  withhold  Information  ("what  the  hell,  they  can't  fix 
It  anyway");  they  offer  only  sketchy  descriptions,  glib  write-ups,  or  none  at 
all;  some  blame  nonmalfunctioning  equipment  for  their  own  deficiencies;  some 
abuse  the  equipment  and  yet  are  willing  to  blame  the  maintenance  man  for  its 
breakdown.  The  operator-malntalner  Interaction  deserves  attention. 

The  human  factors  specialist  would  like  to  have  as  complete  an  inventory 
as  possible  of  the  maintenance  population's  basic  skills,  the  aptitudes  of  its 
members,  and,  as  Dr.  Modrick  said,  perhaps  a  count  of  how  many  are 
left-hemispheric  and  how  many  are  right-hemispheric  with  respect  to  central 
nervous  system  functions.  How  many  are  procedures-oriented  and  how  many  are 
prlnclples-orlented?  What  are  their  short-term  and  long-term  memory 
capabilities?  Can  they  assimilate  more  from  three-dimensional  drawings  than 
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they  can  from  verbal  description?  What  Is  the  optimal  mix  of  verbal  and 
illustrative  materials  for  this  population? 


Task  analysis  can  provide  information  that  relates  to  some  of  the  above 
issues.  We  have  found  summary  classification  of  behaviors,  such  as  that  shown 
in  Table  3,  helpful  in  the  Interpretation  of  the  results  of  task  analyses 
(Berliner,  Angel  A  Shearer,  1964). 


We  find  that  a  very  complete  task  analysis  can  be  developed  by  the  use  of 
video  cameras,  the  General  Physics  Performance  Measurement  System  (essentially 
a  system  for  recording  all  motor  responses),  and  a  record  of  all 
communications.  Unfortunately,  cognitive-type  activities  usually  have  to  be 
inferred  from  overt  activities  and  communications,  supplemented  by  interviews 
with  those  being  observed. 


Finally,  the  characteristics  of  populations  such  as  the  maintenance 
population  of  the  U.S.  Navy  are  probably  always  changing.  Perhaps  there  are 
ways  of  forecasting  what  those  characteristics  will  be  eight  to  ten  years  into 
the  future  when  a  system  that  is  now  in  the  conceptual  stage  will  be 
operational.  If  there  are  such  methods,  I  am  not  familiar  with  them.  But  we 
do  have  moderately  good  methods  for  predicting  what  skills  will  be  needed  to 
maintain  a  system  some  time  before  it  is  operational  and  we  should  exercise 
these.  The  forecasting  of  skill  requirements  and  the  development  of  selection 
and  training  programs  to  insure  that  those  skills  exist  in  the  proper  number 
at  the  right  time  is  complementary  to  our  program  of  design  for  maintenance. 


Methods  and  Procedures 


It  has  been  said  that  any  branch  of  science  or  engineering  makes  progress 
largely  as  a  function  of  its  methods.  Human  factors  is  no  exception.  I  will 
briefly  examine  a  sampling  of  the  methods  referred  to  by  participants  of  this 
conference. 


Activity  Analysis 


Activity  analysis  covers  an  entire  family  of  methods  whose  objective  is 
easy  to  state  but  whose  execution  is  difficult.  The  term  simply  refers  to  the 
gathering  (recording)  and  analysis  of  what  people  are  doing.  Modern  methods 
of  picture  taking  (e.g.,  video  cameras),  response  recording  (e.g.,  the  GPC 
Performance  Measurement  System),  and  recording  of  communications  have  made  it 
possible  to  define  stimulus  and  response  conditions  fairly  well  in  almost  any 
situation.  The  nature  and  interpretation  of  cognitive  activities  still  depend 
heavily  on  inference  and  individual  interpretation. 


Task  analysis  is  currently  the  preferred  method  of  activity  analysis,  at 
least  in  DOD.  I  agree,  however,  that  much  work  needs  to  be  done  to  make  task 
analysis  more  useful. 


Using  the  equipment  mentioned  in  the  previous  paragraph,  one  can  usually 
prepare  a  very  satisfactory  task  analysis.  However,  as  Inaba  suggests,  there 
are  problems  associated  with  the  use  of  task  analysis.  These  include  (1)  a 
tendency  to  gather  an  enormous  amount  of  data,  often  without  a  clear  idea  as 
to  how  it  will  be  reduced,  or,  sometimes,  even  why  it  is  being  gathered,  (2) 


167 


.H 


■A 


A 


M 


V 


% 


a 


[•- 

Lv 

W 

fc! 

l- 

is 


■v' 

sv 


y 


interpretation  difficulties  (see  our  earlier  comments  regarding  the  use  of  the 
Berliner  code),  (3)  assurance  that  the  sample  is  representative  of  the 
population.  Someday,  someone  will  do  a  study  on  the  independence  of  task 
elements  and  find,  as  K.  U.  Smith  did  a  generation  ago  with  respect  to 
therbligs,  that  they  are  not  Independent;  that  one,  therefore,  must  at  least 
recognize  that  they  are  not  additive,  and  that  different  orders  of  execution 
yield  different  results.  This  doesn't  Invalidate  the  technique;  it  does 
suggest  care  in  interpretation.  Finally,  as  Dr.  Modrick  observes,  the  proper 
role  of  task  analysis  in  the  design  process  has  yet  to  be  satisfactorily 
defined. 

Failure  Analysis 

There  are  numerous  methods  aimed  at  the  identification  of  error-likely 
loci  in  systems.  Failure  Modes  and  Effects  Analysis  (FMEA)  and  Fault  Tree 
Analysis  (FTA)  are  two  that  have  been  mentioned  at  this  meeting.  I'm  sure 
that  these  are  useful  techniques;  I  would  feel  more  comfortable  if  someone 
would  show  me  data  demonstrating  their  reliability.  One  study  at 
Perceptronics  suggests  that  different  people  obtain  rather  different  results 
when  developing  FTAs.  This  doesn't  mean  that  such  analyses  are  not  useful;  it 
does  mean  that  we  have  an  obligation  to  search  out  and  carefully  define  the 
limitations  of  the  tools  that  we  employ. 

"Go  to  the  Field" 


I  very  much  admire  the  point  of  view  and  approach  as  demonstrated  by 
Theisen  and  his  group,  by  Orlansky,  by  Inaba,  by  Schmitt  and  others.  They 
recognize  that  there  are  enormous  benefits  to  be  derived  from  gathering  data 
under  actual  operational  conditions  or  trying  systematically  to  extract  useful 
information  from  vast  data  stores.  We  often  underestimate  the  Incredible  fund 
of  information  that  exists  "out  there."  In  addition  to  serving  as  a  source  of 
data  for  immediate  application.  It  can  yield  direction  for  future  supporting 
research. 

"Going  to  the  field"  often  means  resorting  to  the  use  of  techniques  such 
as  questionnaires,  interviews,  and  rating  scales.  I'm  happy  to  report  that 
"opinion"  is  becoming  psychologically  respectable  as  a  source  of  data. 
Interestingly,  it  was  the  engineers,  with  techniques  such  as  Delphi  and  the 
Cooper-Harper  scale,  who  drew  attention  to  the  value  of  these  tools. 

Confirmatory  Methods 

When  I  am  confronted  with  a  situation  in  which  it  is  obviously  going  to  be 
difficult  to  obtain  hard,  objective  data,  I  try  to  use  at  least  two  methods  to 
address  the  same  hypothesis.  Thus,  for  example,  if  I  wanted  to  know  how  a 
maintenance  man  spends  his  time  on  a  certain  job,  and  for  some  reason  I  could 
not  take  direct  measurements,  I  might  (1)  ask  a  sample  of  the  maintenance  men 
to  complete  an  "activities"  checklist  and  (2)  interview  another.  Independent 
sample  as  to  the  nature  of  their  job  activities.  If  the  results  from  these 
two  approaches  agree,  one  can  be  reasonably  sure  that  he  has  reliable  data, 
which  is  always  a  reason  for  rejoicing. 
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Prediction  of  Maintainability 

Towne  gave  an  interesting  paper  on  maintenance  prediction  techniques.  His 
time-synthesis  technique  reminds  me  of  earlier  industrial  engineering  standard 
times  systems  which  have  enjoyed  considerable  success  in  industry.  I  would 
urge  that  any  experimental  work  carried  out  in  this  area  should  give  special 
attention  to  selection  of  the  samples  of  subjects.  They  should  be 
representative  of  U.S.  Navy  maintenance  men  and  not  college  sophomores.  I 
hope,  further,  that  our  dedication  to  the  time  criterion  will  not  cause  us  to 
neglect  the  total  job  context,  including  such  factors  as  job  enrichment.  To 
paraphrase  v/hat  someone  said,  "design  a  system  that  fools  can  maintain  and 
only  fools  will  want  to  maintain  it."  We  don't  want,  or  need,  a  repetition  of 
the  early  job  simplification  work  of  Taylor  and  the  Gilbreths.  (Towne  did  not 
imply  this;  I  just  want  to  be  sure  that  we  keep  it  in  mind  if  and  when  we 
restructure  maintenance  jobs.) 

Hsu  presented  an  excellent  assessment  on  prediction  of  maintainability. 

In  addition  to  warning  us  about  problems  of  additivity  and  sequence,  I  find 
myself  in  complete  agreement  with  him  on  the  limited  usefulness  of  time,  at 
least  as  currently  measured,  as  a  criterion.  It  is  Insensitive  to  design 
variables,  downtime  is  contaminated  with  factors  over  which  the  maintenance 
man  has  no  control,  etc.  We  must  refine  the  traditional  time  measures  if  we 
have  any  hope  of  ever  using  them. 

I  must  address  Hsu's  criticisms  of  techniques  such  as  the  RCA  technique. 

It  was  fully  realized  when  it  was  adopted  that  the  technique  had  many 
imperfections;  however,  many  felt  that  it  was  the  best  available,  or  even  the 
only  game  in  town.  It  is  too  bad  that  more  effort  has  not  been  expended  to 
correct  its  deficiencies. 

Risk  Assessment/Cost-Effectiveness 


Orlansky  for  years  has  been  in  the  forefront  of  those  urging  others  in  the 
human  factors  field  to  express  their  results  in  terms  of  risk  assessment  and 
cost-effectiveness.  I  can  only  urge  that  more  of  us  heed  his  admonitions. 
Orlansky  is  also  a  proponent  of  the  cross-discipline  approach  to  human  factors 
problems.  Again,  heed  him--a  wise  man  with  much  experience.  The  very 
complexity  of  the  maintenance  problem  assures  that  no  one  engineering  or 
scientific  specialty  will  have  the  answer  to  more  than  a  small  piece  of  the 
problem.  The  cross-discipline  approach  breeds  synergism.  This  area  needs 
that. 


Diagnostics 

Inaba  suggests  that  as  equipment  becomes  more  automatic,  diagnosis  will 
become  more  difficult.  Certainly,  unless  we  devote  more  thought  and  effort  to 
the  problem,  Inaba  Is  correct.  Consider,  for  example,  that  with  automatic 
equipment  one  source  of  information  regarding  malfunctions  (the  operator)  is 
completely  missing.  Man-machine  "intimacy"  is  greatly  reduced. 

Inaba’ s  prediction  was  verified  by  Rouse  who  found  that  when  automatic 
equipment  failed,  the  maintenance  men  had  no  idea  why.  Under  such  conditions, 
they  often  make  matters  worse  by  doing  counterproductive  things.  It  suggests 
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to  me  that  In  the  era  of  automaticity  the  diagnostician  (perhaps  a  person 
different  than  he  who  fixes  it)  will  have  to  have  a  better  understanding  of 
the  system  than  has  often  been  the  case  in  the  past. 

I  have  gathered  evidence  which  convinces  me  that  in  many  jobs 
approximately  70  percent  of  the  maintenance  man's  job  is  spent  trying  to 
determine  precisely  what  is  wrong  with  the  equipment  that  he  is  attempting  to 
maintain.  This  requires  a  sensitivity  to  subtle  cues,  a  familiarity  with 
software  (the  maintenance  man  of  the  future  almost  certainly  will  have  to  be  a 
software  expert),  and  an  ability  to  integrate  information  from  a  number  of 
sources.  The  locus  of  this  activity  in  Figure  2  is  "brain  and  spinal  cord." 
And,  let's  face  it,  that  is  the  one  box  of  the  five  shown  in  Figure  2  about 
whose  functioning  we  know  least. 

The  diagnostic  part  of  the  maintenance  man's  job  is  more  like  that  of  the 
veterinarian  than  the  physician.  In  neither  the  case  of  the  maintenance  man 
nor  the  veterinarian  do  the  patients  say  much  to  the  individuals  who  are 
trying  to  help  them.  As  human  factors  specialists,  is  it  not  our 
responsibility  to  see  that  equipment  communicates  its  problems  more 
effectively  to  the  maintenance  man,  preferably  before  irreversible  damage  or 
complete  breakdov/n  have  occurred? 

This  latter  requirement,  essentially  that  of  preventative  maintenance 
(recall  my  earlier  statement  that,  on  the  average,  corrective  maintenance 
costs  three  times  as  much  as  preventative  maintenance),  means  that  patterns  of 
symptoms  will  become  more  important  than  individual  symptoms  and  that  rates  at 
which  things  are  changing,  or  derivative  information,  will  be  very  helpful. 
This  means  also  that  the  feedback  provided  by  the  equipment  must  be  more  and 
more  specific— "operating  or  not  operating"  is  no  longer  specific  enough. 
Rather,  "I'm  operating  but  notice  that  my  temperature  is  rising  rapidly  and  my 
bearings  are  beginning  to  chatter.  Better  check  my  oil'."  (Parenthetically,  I 
observe  that  we  have  been  rather  sloppy  with  respect  to  our  demands  for 
feedback.  In  learning  situations,  for  example,  we  often  assume  that  as  long 
as  the  person  is  getting  some  kind  of  feedback  [a  letter  grade,  for  example] 
that  everything  is  O.K.  We  must  insist  that  the  feedback  be  specific  and  be 
intimately  related  to  the  nature  of  the  action  we  expect  from  the  person.  A 
yellow  caution  light  should  be  just  one  of  a  number  of  feedback  communications 
from  equipment  to  operator  or  maintenance  man.) 

The  things  just  described  will  enable  the  maintenance  expert  to  make  a 
prediction  as  to  what  will  happen  and  when  it  will  happen  if  certain  actions 
are  not  taken.  Sometimes  these  patterns  of  symptoms  will  be  so  complex  that 
It  is  unreasonable  to  expect  the  average  line  maintenance  man  to  make  the 
correct  diagnosis.  (That  he  often  doesn't  is  evidenced  by  the  fact  that 
nearly  50  percent  of  the  time  commercial  aircraft  maintainers  remove  a 
nonfaulty  part  because  of  the  faulty  diagnosis  [Burrows  -  personal 
communication].  I  need  not  remind  you  what  the  actions  of  removal  and 
replacement  themselves  eventually  can  do  to  reliability.)  Oust  as  there  are 
expert  diagnosticians  in  medicine,  so  should  there  be  in  maintenance.  I 
understand  that  at  least  one  robotics  company  has  a  team  of  expert 
diagnosticians  in  the  home  office  who  can  be  called  toll-free  at  any  time 


regarding  the  meaning  of  certain  symptoms  that  the  robot  is  evincing.  Might 
not  the  Navy  designate  specialists  in  diagnostics  who  would  be  available  to 
assist  the  line  maintenance  men? 

All  of  this  leads  one  to  propose  a  sort  of  "strategy  for  curing  sick 
systems."  Its  steps  appear  to  run  somewhat  along  the  following  lines: 

1.  initial  maintenance  requirements  established  during  systems  analysis 

2.  proper  functions  allocation 

3.  interface  design  (otherwise  we  will  have  a  garbage  in— garbage  out 
situation) 

4.  selection  of  people  with  proper  aptitudes  and  attitudes 

5.  training  with  emphasis  on  troubleshooting  (as  it  is  now,  most 
maintenance  men  troubleshoot  in  a  haphazard  manner  or  in  some  favorite  way 
that  they  have  picked  up  somewhere  but  which  may  be  quite  inappropriate) 

6.  constant  sharpening  of  diagnostic  skills  (one  stuc|y  showed  that 
skilled  maintenance  men  sample  with  a  purpose  while  the  unskilled  sample  and 
then  try  to  decide  v/hat  to  do) 

7.  proper  manual  and  job-aids  design 

8.  reasonable  environmental  conditions 

9.  machines  that  communicate  their  ills  to  man 

10.  strong,  active  management  support  of  the  maintenance  program. 

What  Is  a  Maintenance  Error? 

Although  I  have  heard  frequent  reference  to  the  term  at  this  conference,  I 
cannot  help  but  observe  that  even  in  this  sophisticated  group,  no  one  has 
defined  it'.  I  would  think  that  a  clear  definition  of  error,  plus  a 
classification  of  types  and  number  of  errors,  would  be  most  useful.  For 
example,  if  a  maintenance  job  takes  four  times  as  long  to  perform  as  it 
should,  is  this  an  error?  Is  an  error  that  is  discovered  and  corrected  before 
the  job  is  completed  an  error? 

If  systematic  errors  are  discovered  in  acts  such  as  calibration,  they  can 
be  corrected  by  the  application  of  constants.  Random  errors  can  be  corrected 
to  some  extent  by  redundancy. 

As  Singleton  has  suggested,  one  looks  for  consistency  in  such  matters,  not 
simply  rationality.  It  is  clear  that  a  program  directed  toward  reduction  of 
errors  in  maintenance  must  consider  the  same  behavioral  parameters  (admittedly 
from  a  somewhat  different  point  of  view)  as  reduction  of  errors  in 
operation— clear,  unequivocal  inputs  that  facilitate  central  nervous  system 
processing  plus  unambiguous  means  for  making  the  necessary  motor  responses. 
Appropriate  feedback  attesting  to  the  success  or  failure  of  directed  actions 
is  a  necessity  (see  Figure  2). 


Model s 


Good  models  can  lead  to  good  theories  and,  as  is  often  said,  there  is 
nothing  as  practical  as  a  good  theory.  We  have  seen  many  good  examples  of 
modeling  at  this  conference;  perhaps  we  will  see  theories  emerging  from  some 
of  them  in  the  near  future,  or  at  least  adoption  and/or  elaboration  of 
existing  theories  to  accommodate  maintenance  issues. 

It  is  encouraging  to  see  the  support  that  ONR  is  giving  this  area.  I'm 
not  familiar  with  Towne' s  work  on  troubleshooting  strategies  but  I  do  hope 
that  it  takes  account  of  not  only  the  particular  characteristics  of  the  U.S. 
Navy  maintenance  population  but  also  the  differences  within  that  population. 

I  would  be  astonished,  for  example,  if  any  single  troubleshooting  strategy 
proved  to  be  best  for  everyone. 

I  find  Sheridan's  discussion  on  internal /external  models  useful  in 
situations  such  as  this.  You  may  recall  that  Sheridan  suggests  that  the 
greater  the  differences  between  the  internal  model  and  the  corresponding 
external  model,  the  more  elaborate  will  be  the  transfer  functions  between  the 
two.  It  would  appear  profitable  to  make  these  as  similar  as  possible,  but  how 
to  do  it?  Once  maintenance  personnel  are  selected,  it  would  appear  that 
training  would  be  the  primary  resource  available  to  assure  veridical ity 
between  the  internal  and  external  models.  ("Training,"  in  this  context, 
includes  the  materials  used  in  Instruction,  as  well  as  on-the-job  manuals, 
etc.).  The  external  model  is  determined  by  design  features  and  is  clearly 
within  the  province  of  human  factors  engineering.  The  model  validates  the 
close  interrelationship  between  training  and  human  factors  engineering  that 
Professor  Rouse  supported  with  his  remarks. 

Design  for  effective  maintenance  must  follow  the  systems  development 
model.  The  results  of  interviews  that  I  conducted  with  systems  engineers  and 
design  engineers  convince  me  that  most  (70-90  percent)  of  the  major  design 
decisions  (the  "drivers,"  as  they  are  often  termed)  are  made  by  the  end  of  the 
conceptual  stage.  The  implications  of  this  are  clear:  we  will  never  be  in  a 
position  to  optimize  the  external  model  ("optimize"  here  means  maximum 
correspondence  between  internal  and  external  models)  until  we  are  able  to 
specify  design  requirements  at  the  initial  stages  of  equipment  and  systems 
development.  We  have  not  been  able  to  do  this;  perhaps  this  is  the  reason 
that  the  engineers  seem  more  and  more  to  be  going  to  ATE  and  BITE.  (Topmiller 
and  I  have  elaborated  on  something  developed  by  Crawford  and  Altman  that 
appears  in  HEGED.  It  might  help  a  bit  in  this  regard  [see  Figure  3].) 

Finally,  if  the  model  doesn't  fit  carefully  collected  human  performance 
data,  change  the  model'. 


A  Strategy  for  Curing  Ills 

All  of  the  above  leads  me  to  propose  a  strategy  for  curing  maintenance 
ills.  What  might  be  some  of  the  components  of  that  strategy? 


Derivation  of  Requirements  and  Objectives 


1,  Policy  of  Maintenance  Priorities  and  Tradeoffs 

2,  Interface  between  Maintenance  and  Other 
System  Functions 

3,  Critical  Measures  of  Maintenance  Effectiveness 

4,  Impact  of  Maintenance  Performance  System 
Parameters  and  Functions 

5,  Alternative  Maintenance  Objectives  Allowable 
within  Design  and  Maintenance  Requirements 


1.  Management  and  Administrative  Policies 

2.  Long  Range  Plans 

3.  Maintenance  Manpower  Availability 

4.  Facilities 

5.  Time  Frame 

6.  Users  Expectations,  Capabilities 

7.  Interfaces  with  other  Systems  and  Environments 


Definition  of  Maintenance  Levels,  Areas 


1.  Nature  of  Work  Done 

2.  Extent  of  Maintenance  Performed 

3.  Location  of  Work 

4.  Subsystems  Worked  On 


Identification  of  Maintenance  Functions 
and  Work  Flow 
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Adjusting  and  Servicing 
Fault  Correction 


1 

1 

1.  Cost  6.  Type  of  Functions 

2,  Flexibility 

Allocation  of  Maintenance  Functions  - tt* 

3.  Reliability 

4.  Speed 

1 

5.  Available  Manpower 

i 

Establishment  of  Maintainability  Programs 
Plans 


1,  Relations  to  Operator  Objectives 

2,  Cost  Analysis 

3,  Maintenance  Organization 

4,  Personnel  Requirements 

5,  Logistics  and  Supply 

6,  Facilities 

7,  Prime  Equipment 

8,  Support  Equipment 

9,  Auxiliary  Job  Aids 


Steps  In  Idealized  phasing  of  maintainability  design. 

(Source:  Topmiller  and  Christensen,  Maintainability  Assessment 
and  Design  for  Nuclear  Power.  Adapted  from  Crawford  and  Altman  in 
Van  Cott  and  Klnkade,  pp.  588.) 
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Model /Theory  Development 

You  night  feel  that  the  human  factors  problems  associated  with  maintenance 
are  so  pressing  that  we  don't  have  time  to  develop  models  and  theories.  I 
suggest  that  we  don't  have  time  not  to  do  so.  We  have  been  struggling  along 
in  a  piecemeal  fashion  in  this  area  for  at  least  25  years  and,  frankly,  we 
aren't  much  closer  to  a  solution  than  we  were  a  generation  ago.  Let  us  at 
least  take  what  is  relevant  and  available  from  existing  theories  of  human 
performance  and  begin  the  arduous  task  of  relating  these  to  the  development  of 
some  overall  "theory  of  human  performance  in  maintenance."  We  may  be 
surprised  at  what  we  already  know— but  it  does  need  integration. 

Systems  Analysis 

Our  strategy  must  include  a  systems  analysis  whose  products  help  us  as 
early  as  possible  in  the  developmental  cycle  to  identify  maintenance  functions. 

Functions  Allocation 

Once  these  functions  are  identified,  we  must  allocate  them  to  man,  to 
machines,  and  to  combinations  of  men  and  machines.  Task  analyses, 
appropriately  employed,  can  be  of  considerable  help  here. 

Interface  Design 

We  know  a  lot  about  interface  design  that  we  could  apply  to  the 
man-machine  interaction  but  in  the  past  we  seem  to  have  slighted  the 
maintenance  man  in  favor  of  the  operator.  We  cannot,  for  example,  expect 
valid  diagnostics  without  valid  and  appropriate  inputs  into  the  central 
nervous  system  of  the  maintenance  man. 

Workplace  Design 

Virtually  no  attention  has  been  given  to  the  design  of  workplaces  for 
maintenance,  both  within  the  system  and  at  the  bench  level.  I  recall  a  visit 
to  the  RCA  plant  in  Anderson,  Indiana,  where  most  of  the  wiring  and  plumbing 
on  the  machine  tools  had  been  removed  from  the  bowels  of  the  equipment  and 
made  more  accessible  to  the  maintenance  man.  The  effects  on  morale  and 
maintenance  costs  were  Impressive.  This  leads  to  the  next  component  of  our 
strategy. 

Environmental  Considerations 

Planka  reminds  us  of  the  Importance  of  environmental  factors.  We  have 
considerable  knowledge  regarding  the  effects  of  individual 
stressors— temperature.  Illumination,  etc.  Why  aren't  we  as  concerned  about 
those  Instances  where  the  maintenance  man  is  involved,  as  those  where  the 
operators  are  Involved? 

Manuals  and  Other  JPAs 

This  Is  an  area  about  which  we  know  quite  a  bit  and  have  even  applied  some 
of  that  knowledge.  Yet  I  still  see  manuals  with  readability  Indexes  at  the 
1 Oth-1 2th  grade  level,  illustrations  poorly  integrated  with  text,  etc. 
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Machines  Monitor  Men 
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It  v/as  my  mentor's  fond  hope  that,  since  men  are  such  poor  monitors,  we 
would  someday  be  clever  enough  to  design  systems  in  which  machines  would 
monitor  men--"don't  do  this  to  me,  it  will  cause  a  malfunction;"  "you  are 
unduly  stressing  me."  Through  the  use  of  interlocks,  caution  lights, 
record-keeping,  surrogates,  and  the  like,  we  have  made  some  progress  toward 
Paul  Fitts'  goal  but  have  a  long  way  to  go.  Communication  between  equipment 
and  its  maintainers  needs  much  more  work.  Let's  design  hardware  so  that  it 
will  tell  us  when  and  where  it  hurts  so  that  we  can  help  it  before  it  suffers 
a  breakdown.  Outpatient  care  is  much  less  expensive  than  hospitalization. 

Selection  and  Training 
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We  contend  that  we  are  "systems-oriented"  in  our  approach  to  the  solution 
of  problems.  Thus,  we  must  include  selection  and  training  considerations  in 
our  strategy.  We  must  develop  means  for  facilitating  the  examination  of  the 
multitude  of  tradeoffs  involved  in  the  selection  x  training  x  human 
engineering  interaction.  We  have  developed  specialists  among  the  functional 
areas  of  maintenance  (electronic,  mechanical,  etc.);  perhaps  we  should 
consider  the  development  of  specialists  within  functional  areas  (experts  in 
diagnostics,  for  example). 

Organizational  Considerations 

If  nothing  else,  a  firm's  table  of  organization  tells  you  what  functions 
that  firm  considers  significant.  All  of  you  know  industrial  firms  that  have  a 
vice-president  for  operations.  How  many  of  you  know  of  firms  that  have  a 
vice-president  for  maintenance?  I  know  a  few— one  is  United  Airlines,  the 
firm  that  supplied  the  supporting  data  to  which  I  referred  earlier. 


1 

is 


For  far  too  long  the  maintenance  man  has  felt,  and  usually  with 
justification,  that  he  and  his  contributions  were  secondary  to  others  in  the 
system.  One  survey  disclosed  that  maintenance  men  feel  (1)  that  designers 
intentionally,  or  at  least  through  lack  of  attention,  build  in  irritation 
factors  for  maintenance  men,  (2)  that  management  largely  Ignores  maintenance 
men,  and  (3)  that  computers  are  little  more  than  expensive  nuisances.  Look  in 
the  mirror— what  kind  of  support  Is  your  organization  giving  the  maintenance 
man?  In  this  technological  age,  I  can  see  us  gradually  dispensing  with 
operators  for  many  functions;  I  cannot  see  us  dispensing  with  maintenance  men. 

The  procuring  organization  (SPO?)  has  a  responsibility  to  develop  a 
genuine  capability  on  its  side.  Otherwise  adequate  assessment  of  the 
offeror' s  product  from  the  standpoint  of  maintainability  is  impossible. 


Final  Element  of  Our  Strategy 


As  I  review  the  elements  of  our  strategy  for  curing  maintenance  ills,  one 
item  pervades  my  thoughts— yes,  we  need  better  models  and  theory,  but,  as 
Orlansky  and  Inaba  pointed  out,  there  is  an  enormous  amount  of  information 
already  available  that  we're  not  applying.  As  Orlansky  said,  "Let  us  document 
our  successes  and  sell  them  In  terms  of  readiness,  manpower  and 
cost-effectiveness."  And  let  us  also  heed  Inaba' s  admonition  to  the  effect 
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that  we  must  be  prepared  to  pay  for  maintainability  in  our  systems.  But  let's 
be  sure  that  when  management  is  ready  to  listen  we  have  our  product 
well-defined  so  we  can  offer  a  meaningful,  well -integrated  package. 


And,  finally,  let's  open  up  the  Vies  of  communication,  not  only  with 
management,  but  also  with  the  maintenance  men  themselves.  Theisen  and  his 
associates  have  given  us  a  taste  of  the  good  things  that  can  result  from  field 
records.  Let's  encourage  means  for  Increasing  the  effectiveness  of 
communications  between  operators  and  maintenance  men.  When  you  take  your 
automobile  into  the  garage  for  repair,  you  can  help  the  mechanic  a  great  deal 
by  describing  the  symptoms  that  you  have  observed.  Yet  I  have  had  pilots  tell 
me  that  there  is  no  sense  in  writing  a  detailed  description  of  observations  of 
incipient  or  actual  malfunctions  because  "they  can't  fix  it  anyway."  What  a 
sad  commentary  on  the  quality  of  communications  between  operators  and 
maintenance  men. 

(As  I  am  writing  this,  a  fellow  worker,  John  Howard,  put  an  advertisement 
in  front  of  me  in  which  the  headline  states,  "With  Digital's  Remote  Diagnosis, 
we  know  what's  wrong  with  your  computer  before  we  get  there."  That's  what  I'm 
talking  about'.) 
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Criteria 


No  one  would  question  the  need  for  adequate  criteria  in  the  area  of  design 
for  maintenance.  What  do  we  mean  by  "adequate  criteria"?  As  a  minimum,  the 
criteria  must  be  reliable,  valid,  and  sensitive.  The  most  commonly  used 
criteria  in  the  past  have  been  time-related:  MTBF,  MTTR,  etc.  These  measures 
may  be  reasonably  reliable,  perhaps  valid  (at  least  for  some  purposes),  but 
their  sensitivity  to  human  factors  parameters  must  be  questioned.  As  a 
minimum.  It  would  appear  that  time  measures  need  to  be  broken  down  into  their 
components  since  the  gross  time  figures  appear  to  be  overwhelmed  by  matters 
over  which  the  Individual  maintenance  man  has  little  or  no  control. 

Errors  constitute  good  criteria  but  there  are  difficulties  here  also. 

They  are  difficult  to  record,  their  definition  often  causes  difficulty, 
self-incriminating  errors  are  seldom  reported,  and,  as  stated  previously, 
occasionally  malfunctions  that  never  happened  are  reported. 
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Other  criteria,  then,  are  cost  and  unnecessary  replacement  of  parts, 
can  Imagine  the  difficulties  associated  with  obtaining  reliable,  valid, 
sensitive  information  on  the  human  factors  aspects  of  such  criteria. 


One 
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Lacking  good  field  criteria  for  assessing  the  effectiveness  of  training, 
we  have  adopted  the  time-honored  procedure  of  using  school  achievement  and 
performance  on  the  job.  Orlansky  reminds  us  that  a  positive  correlation 
between  these  two  variables  has  yet  to  be  established.  We  must  remind 
ourselves,  however,  that  this  does  not  mean  that  the  training  program  is  of  no 
value.  The  lack  of  correlation  could  be  a  reflection  of  (1)  the  lack  of 
reliable  field  criteria,  (2)  the  lack  of  reliable  classroom  criteria,  (3)  the 
lack  of  any  positive  relationship  even  with  reliable  criteria,  or  (4)  all  of 
the  above. 
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Orlansky  has  shown  the  way  to  begin  to  develop  improved  criteria  with  his 
plea  for  more  and  better  quantification  and  exploitation  of  what  is  already 
available.  The  accident  and  mishap  data  base  that  Schmitt  and  his  associates 
are  developing  looks  promising. 


Concluding  Remarks 


The  goal  is  clear;  the  way  is  not.  All  of  us  know  what  we  would  like  to 
achieve;  namely,  the  means  for  developing  a  maintainability  program  that,  when 
Implemented,  would  contribute  maximally  to  the  effectiveness  of  that 
system— effectiveness  to  be  determined  by  measurement  in  terms  of  the  criteria 
specified  by  the  systems  manager  and  systems  engineer. 

LT  McBride  is  proceeding  on  a  broad  front  to  satisfy  these  requirements 
and  that  is  necessary.  Excellent  work  has  been  reported  here;  it  is  his  very 
difficult  and  important  job  to  see  how  these  pieces  fit  together.  The  very 
process  of  attempting  to  fit  them  together  will  disclose  those  points  at  which 
gaps  in  the  program  exist  and/or  where  emphasis  may  have  been  misplaced.  I 
know  that  I  speak  for  everyone  here  when  I  say  that  I  sincerely  hope  that 
these  proceedings  have  helped  him  a  bit  with  that  difficult  assignment. 
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APPENDIX  A 


Twenty-four  people  responded  to  the  survey  conducted  at  the  Design-for- 
Maintainers  Conference,  held  in  Pensacola  Beach,  Florida,  9-11  March  1982. 
Seventeen  were  psychologists,  five  were  engineers,  and  two  were  maintenance 
people. 

There  is  a  total  of  376  years  of  human  factors  experience,  with  a  mean  of 
about  15  years.  One  hundred  and  thirty-seven  are  in  Design  (mean  about  8), 
seventy-five  in  Training  (mean  about  6),  thirteen  in  Selection  (mean  about  3), 
one  hundred  and  thirty  in  Application  (mean  about  9)  and  ninety-one  in 
Technical  Development  (mean  about  8). 

1.  Assuming  a  "triad  model,"  what  relative  emphases  are  presently  applied 
to  the  following  elements?  Rate  your  choices  on  the  ten-point  scales 
provided,  such  that  they  add  to  ten J 


Mean  (x)  Range 


Low 

High 

Design 

1  4.5  | 

2  1 

3 

Training 

1  3.7  | 

1  1 

I  7 

Selection 

ITT  r 

1  1 

6 

2.  What  is  your  perception  of  our  historical  emphasis  in  each  of  these 
three  areas?  (  T.  -  10) 


Mean  (x)  Range 

Low  High 


Design 

Training 

ITS - 1 

I 

8 

|~ TO  p- 

1  1 

"9 

Sel ection 

i  m  i 

1  1 . 

7 

What  is  the  ideal 

investment  of  emphasis?  (  z  =  10) 

Mean  (3T) 

Range 
Low  Hi 

9h 

Design 

1  5.8  | 

“175  I 

8 

Training 

1  573  | 

1  1 . 

8 

Sel ection 

\~T7S  r~ 

1  ( . 

8 

4.  What  future  investments  emphasis  do  you  predict  will  actually  be  made 
in  these  three  areas?  (  z  =  10) 


Mean  (x)  Range 

Low  High 


Design 

i~ *77 — r 

— 2 — r 

8 

Training 

1  378  r 

2  i 

9 

Selection 

1  275  r 

1  1 

9 

1  The  rationale  for  the  forced  cumulative  scheme  Is  that,  emphasis,  defined 
here  as  money,  is  assumed  finite  and  Inelastic.  If  some  investment, 
therefore,  Is  divided  among  three  Independent  recipients,  the  sum  must  persist 
In  adding  to  the  original,  NIF  and  Introductory  questionnaire  theory 
notwithstanding. 


h\Y 


5.  With  respect  to  reliability  and  maintainability,  give  your  opinion  as 
to  the  percentage  of  emphasis  typically  given  to  each  approach  during  the 
design  phase  of  equipment  evolution?  (£  =  100%) 


* 


Mean  (x) 

Range 

Low  High 

i"  * 

Reliabil ity 

1  '  71V?  '  | 

“  In- 1  Tfi— i 

Maintainabil  ity 

1  20  T 

TT5  |  5CT~| 

6. 

For  the  future,  where 

do  you  predict  the  emphasis  will  actually  be 

V, 

placed? 

{  Z =  100%) 

M* 

Mean  (x) 

Range 

•  h 

Low  High 

N* 

Reliabil ity 

I” 5571  r 

"“TO  |  To“| 

Maintainability 

1  40.0  T 

~2(5  |  W~\ 

of  current  Built-in-Test  (BIT) 

u 

7. 

What  is  your  opinion 

technology? 

extremely  effective: 

0 

• 

very  effective: 

2 

*/ 

somewhat  effective: 

15 

ineffective: 

6 

n 

very  ineffective: 

1 

.  . j 

8. 

What  is  your  opinion  of  BIT's  future? 

V» 

extremely  effective: 

3 

very  effective: 

11 

□ 

somewhat  effective: 

8 

ineffective: 

1 

\’v 

very  ineffective: 

1 

9.  In  which  of  the  following  should  we  invest  ourselves?  (£  =  100%) 


Diagnostics  *  60% 

Remove/Repair/Replace  «  40% 

10.  Rate  on  the  ten-point  scale  your  feelings  regarding  the  importance  of 
Robotics  for  maintenance  in  the  future,  x  =  5 


■-1 


■Vi 


RD-A160  934 
UNCLASSIFIED 


PROCEEDINGS  OF  A  CONFERENCE  ON  A  COMPREHENSIVE  2/2 

TECHNICAL  REVIEW  OF  HUMAN  OJ>  NAVAL  AIR  DEVELOPMENT 
CENTER  WARMINSTER  PA  D  K  MCBRIDE  ET  AL  11  MAR  82 
SBI-AD-F630  624  F/G  5/5 


NL 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUREAU  OF  STANDARDS-1963-A 


AGENDA 


TUESDAY,  9  MARCH 
0830  -  0900 
0900  -  0930 

0030  -  0945 

0945  -  1030 

1030  -  1100 
1100  -  1200 
1200  -  1330 
1330  -  1400 

1400  -  1445 

1445  -  1515 
1515  -  1600 

1600  -  1615 

1830  -  1930 


DESIGN  FOR  MAINTAINERS  CONFERENCE 
9-11  March  1982 

Holiday  Inn  Convention  Center  by  the  Sea 
Pensacola  Beach,  Florida 


Registration 

A  Human  Factors  Design.for-Maintainers  Technology 
Development  Program 
D.  McBride,  J.  Lambert 

(Naval  Air  Development  Center/Eagle  Technology,  Inc.) 

An  Overview  of  ONR  Human  Factors  Research  Programs 
G.  Maleckl 

(Office  of  Naval  Research) 

Developing  Solutions  to  Problems 

K.  Inaba,  F.  Fuchs 

(XYZYX  Information  Corporation) 

Coffee  Break 

Developing  Solutions  to  Problems  (Cont.) 

LUNCH 

The  Performance  of  Maintenance  Technicians  on  the  Job 
J.  Orlansky,  J.  String 
(Institute  for  Defense  Analyses) 

Derivation  of  Maintainability  Design  Guidelines: 

An  Experimental  Approach 
C.  Thelsen,  S.  Hsu 
(Essex  Corporation) 

Coffee  Break 

A  Survey  of  Methodological  Issues  In  Maintainability  Research 
S.  Hsu,  C.  Thelsen 
(Essex  Corporation) 

Evaluating  the  Effectiveness  of  Maintenance  Training 
by  Using  Currently  Available  Maintenance  Data 
J.  String,  J.  Orlansky 
(Institute  for  Defense  Analyses) 

Social  Hour  -  hors  d' oeuvres,  limited  open  bar 


WEDNESDAY,  10  MARCH 


0800  -  0830  Coffee 

0830  -  0930  Diagnostic  Psychological  Issues  and  Maintenance 

Design  Features 

D.  Towne 

(Behavioral  Technology  Laboratories, 

University  of  Southern  California) 

0930  -  1000  Human  Factors,  System  Safety,  and  Aviation  Maintenance 

M.  Pianka 

(Naval  Air  Test  Center) 

1000  -  1030  Coffee  Break 

1030  -  1100  Design  for  Effective  Maintenance  (DEM) 

J.  Schmitt,  L.  Lamb,  J.  Mocharnuk 
(Harris  Corporation) 

1100  -  1200  Design,  Implementation,  and  Evaluation  of  Approaches 

to  Improving  Maintenance  Through  Training 
W.  Rouse 

(Georgia  Institute  of  Technology) 

1200  -  1330  LUNCH 

1330  -  1415  Cost-Effectiveness  of  Mai ntenance  Simulators  for 
Military  Training 
J.  Orlansky,  J.  String 
(Institute  for  Defense  Analyses) 

1415  -  1445  Biodynamic  Modeling  of  the  Maintainer 

E.  Winkler,  J.  Miller 
(McDonnell -Douglas  Corporation) 

1445  -  1515  Coffee  Break 

1515  -  1545  Using  Design  for  Maintainer  Technology:  Lessons  Learned 

T.  Jones,  D.  Mahar 
(Pacific  Missile  Test  Center) 

1545  -  1615  Comparing  Engineering  Psychology  and  Industrial  Engineering 

Approaches  to  Logistic  Problems 
R.  Keesee 

(University  of  Louisville) 

1615  -  1645  A  Critical  Review 

J.  Christensen 

(General  Physics  Corporation) 


THURSDAY,  11  MARCH  (closed  session) 


0800  -  0830 

Coffee 

0830  -  1000 

Review  Panel 

1000  -  1030 

Coffee  Break 

1030  -  1200 

Review  Panel 

(Cont.) 

1200  -  1330 

LUNCH 

1330  -  1500 

Review  Panel 

(Cont.) 

1500  -  1530 

Coffee  Break 

1530  - 

Review  Panel 

(Cont. ) 

Conference  On  DesIgn-for-Maintainers 
9-11  March  1982 

ATTENDEE  LIST 


Dr.  Will  Ian  B.  Askren 
AFHRL/LRL 

Wrlght-Patterson  AFB,  OH  45433 
Commercial  (513)  255-3771 
Autovon  785-3771 


Ms.  Diana  Barry 
Rowland  A  Co. 

6961  Hanging  Moss  Rd. 
Orlando,  FL  32807 
Commercial  (305)  677-6951 


Dr.  Jullen  Christensen 
General  Physics  Corporation 
1010  Woodman  Drive,  Suite  240 
Dayton,  OH  45432 
Commercial  (513)  254-6707 


Dr.  William  H.  Crooks 
Perceptronlcs,  Inc. 

6271  Varlel  Avenue 
Woodland  Hills,  CA  91367 
Commercial  (213)  884-7470 


LCDR  Larry  M.  Dean 
CNO  (OP-11 5E) 

Washington,  DC  20350 
Commercial  (202)  694-4970 
Autovon  224-4970 


Dr.  Floyd  Glenn 

Analytics 

2500  Maryland  Rd. 

Willow  Grove,  PA  19090 
Commercial  (215)  657-4100 


Mr.  John  F.  Hayes 
US  Amy  Research  Institute 
ATTN:  PERI-II  (Mr.  Hayes) 
5001  Elsenhower  Ave. 
Alexandria,  VA  22333 
Commercial  (202)  274-8695 
Autovon  284-8695 


*  See  Distribution  List  for  Current  Address 


Ms.  Laura  Hitchcock 
Essex  Corporation,  Suite  200 
65  West  Street  Road 
Warminster,  PA  18974 
Commercial  (215)  674-8710 


Mr.  Wayne  Hostetter 
Allen  Corp.  of  America 
3751  Maguire  Blvd. 

Suite  270 
Orlando,  FL  32803 
Commercial  (305)  896-9300 


*  Dr.  Shang  Hwa  Hsu 
Essex  Corporation 
65  West  Street  Road 
Suite  200 

Warminster,  PA  18974 


Dr.  Kay  Inaba 
XYZYX  Information  Corp. 
21116  Vanowen  Street 
Canoga  Park,  CA  91303 
Commercial  (213)  883-8200 


Mr.  Paul  Kaster 
Eagle  Technology,  Inc. 

2300  South  9th  St.,  Suite  400 
Arlington,  VA  22204 
Commercial  (703)  979-8300 


*  Dr.  Robin  Keesee 
Speed  Scientific  School 
University  of  Louisville 
Louisville,  KY  40208 


Mr.  Harland  Ki dwell 

Singer  Link  Flight  Simulation  Division 
2224  Bay  Area  Blvd. 

Houston,  TX  77058 
Commercial  (713)  488-5510 


:s  *»  ■ 


V 

& 


£ 


m 


K 


¥ 


ri 

w] 

I 


R 

I* 


m 


I 


193 


Mr.  Larry  C.  Lamb 

Harris  Corporation 

Gov't  Electronics  Systems  Div. 

P.0.  Box  37 

Melbourne,  FL  32901 

Commercial  (305)  729-7127 

*  Or.  Joseph  V.  Lambert 
Eagle  Technology,  Inc. 

320  West  Street  Road 
Warminster,  PA  18974 

*  LT  Dennis  McBride 

Naval  Air  Development  Center 
Code  6021 

Warminster,  PA  18974 
Commercial  (215)  441-2561 
Autovon  441-2561 

Dr.  L.  Bruce  McDonald 
McDonald  A  Associates,  Inc. 

988  Woodcock  Rd. 

Suite  136 
Orlando,  FL  32803 
Commercial  (305)  896-8539 

Mr.  Dale  E.  Mahar 
Pacific  Missile  Test  Center 
Human  Factors  Engineering  Branch 
Code  1226 

Point  Mugu,  CA  93042 
Commercial  (805)  982-8981 
Autovon  351-8981 

Mr.  Gerald  S.  Maleckl 
Code  442  EP 

Office  of  Naval  Research 
800  N.  Quincy  Street 
Arlington,  VA  22217 
Commercial  (202)  696-4741 
Autovon  226-4741 

*  LCDR  T.  Mitchell 
Crew  Systems  Division 
Naval  Air  Systems  Command 
Washington,  DC  20361 


Dr.  John  B.  Mocharnuk 

Harris  Corporation 

Government  Electronics  Systems  Di 

P.0.  Box  37 

Melbourne,  FL  32901 

Commercial  (305)  727-6131 

Dr.  Tony  Modrick 
Honeywell,  Inc. 

MN  17-2314 
2600  Ridgway  Parkway 
Minneapolis,  MN  55440 
Commercial  (612)  378-5016 

Mrs.  Louida  D.  Murray 
Eagle  Technology,  Inc. 

320  West  Street  Road 
Warminster,  PA  18974 
Commercial  (215)  672-6250 

Mr.  Gregory  J.  Opetosky 
Lund  Consulting,  Inc. 

1903  E.  Main  St. 

P.0.  Box  315 

Mohegan  Lake,  NY  10547 

Commercial  (914)  528-870* 

Dr.  Jesse  Orlansky 
Institute  for  Defense  Analyses 
1801  N.  Beauregard  Street 
Alexandria,  YA  22311 
Commercial  (703)  845-2293 

Mr.  William  J.  Phillips 
Computer  Sciences  Corporation 
6565  Arlington  Blvd. 

Falls  Church,  VA  22046 
Commercial  (703)  237-2000 

tCDR  M.  J.  Pianka 
Code  SY  50 
SETD 

Naval  Air  Test  Center 
Patuxent  River,  MD  20670 
Commercial  (301)  863-4146 
Autovon  356-4146 


*  See  Distribution  List  for  Current  Address 


Mr.  Joseph  Pontecorvo 
Federal  Aviation  Administration 
Aircraft  Maintenance  Division 
800  Indepedence  Avenue,  SW 
Washington,  DC  20591 
Commercial  (202)  426-3546 
Autovon  426-3546  (FTS) 

Mr.  William  Rizzo 
Naval  Training  Equipment  Center 
Code  N— 71 2 
Orlando,  FL  32813 
Commercial  (305)  646-5130 
Autovon  791-5130 

Dr.  William  B.  Rouse 

Center  For  Man-Machine  Systems  Research 

Georgia  Institute  of  Technology 

Atlanta,  GA  30332 

Commercial  (404)  894-3996 

Mr.  Curt  Sandler 
NAVAIRSYSCOM 
AIR-531 3D 

Washington,  DC  20361 
Commercial  (202)  692-7486/7 

Dr.  John  C.  Schmitt 

Harris  Corporation 

Gov't  Electronics  Systems  Dlv. 

P.0.  Box  37 
Melbourne,  FL  32901 
Commercial  (305)  729-7127 

Dr.  C.  J.  Thelsen,  Jr. 

Essex  Corporation 
65  West  Street  Road 
Suite  C-200 
Warminster,  PA  18974 
Conmerclal  (215)  674-8710 

Mr.  Rules  Tinsley 

Federal  Aviation  Acknlnl  strati  on 

800  Independence  Ave. 

Washington,  DC  20591 
Commercial  (202)  426-9472 

Dr.  Douglas  M.  Towne 

Behavioral  Technology  Laboratories 

1845  South  Elena  Avenue 

Fourth  Floor 

Redondo  Beach,  CA  90277 

Commercial  (213)  540-3654 


Mr.  Martin  A.  Traub 
Appll-Mation,  Inc. 

6955  Hanging  Moss  Rd. 

Orlando,  FL  32807 
Commercial  (305)  678-6955 

Mr.  Edward  R.  Winkler 
McDonnell  Douglas  Corporation 
Dept  E423,  Bldg  271  C/1 /MSI 68 
P.0.  Box  516 
St.  Louis,  MO  63166 
Commercial  (314)  233-7096 

Mr.  T.  J.  Wong 
Douglas  Aircraft  Co. 

MS  35-36 

3855  Lakewood  Blvd. 

Long  Beach,  CA  90846 
(213)  593-8641 


Design-for-Malntalners  Conference 
Conference  Proceedings 
Distribution  List 


Dr.  Earl  Alluisl 
Chief  Scientist 
HQ,  AFHRL 

Brooks  AFB,  TX  78235 
'.Commercial  (512)  536-3605 

Mr.  P.  J.  Andrews 
Code  03416 

Naval  Sea  Systems  Command 
Room  880,  Crystal  Plaza  6 
Washington,  DC  20362 
'.AV  222-9760/1/2/4 
'.Commercial  (202)  692-9760/1/2/4 

CDR  James  H.  Ashburn 

Naval  Aerospace  Medical  Research  Lab 

Pensacola,  FL  32508 

Dr.  William  Askren 
AFHRL/LRL 

Wrlght-Patterson  AFB,  OH  45433 
'.AV  785-3771 

'.Commercial  (513)  255-3771 

Ms.  Diana  Barry 
Rowland  A  Co. 

6961  Hanging  Moss  Rd. 

Orlando,  FL  32807 
Commercial  (305)  677-6951 

Dr.  Robert  E.  Blanchard 
M-8  Montlcello  Street 
Del  Mar,  CA  92014 

Lt.  Col.  Joseph  Blrt 
AFHRL-LR 

Wrlght-Patterson  AFB,  OH  45433 
'.AV  785-3713 

'.Commercial  (513)  255-3713 

Dr.  Richard  Bromberger 

Naval  Material  Command  (MAT  08D4) 

Crystal  Plaza  5 

Washington,  DC  20360 

'.AV  222-2164 

'.Commercial  (202)  692-2164 


Mr.  Gerald  Chaikin 

Human  Engin.  Lab.  Detachment 

DRXHE-MI 

US  Army  Missile  Command 
Redstone  Arsenal ,  AL  35898 
'.AV  746-2048 

'.Commercial  (205)  876-2048 

CAPT.  Paul  R.  Chatelier 
OUSDRAE 

Room  3B129  Pentagon 
Washington,  DC  20301 
'.AV  225-9777 

'.Commercial  (202)  695-9777 

Dr.  Julien  Christensen 
General  Physics  Corp. 

1010  Woodman  Ave. 

Dayton,  OH  45432 
'.Commercial  (513)  254-6707 

Dr.  William  H.  Crooks 
Perceptronlcs,  Inc. 

6271  Varlel  Avenue 
Woodland  Hills,  CA  91367 
Cocnerclal  (213)  884-7470 

LT  Thomas  N.  Crosby 
Naval  Training  Equipment  Center 
Code  N712 
Orlando,  FL  32813 
'.Commercial  (305)  646-5130 

CDR  Mike  Curran 
Code  270 

Office  of  Naval  Research 
800  N.  Quincy  St. 

Arlington,  VA  22204 
'.AV  220-4713 

'.Commercial  (202)  696-4713 

LCDR  Larry  M.  Dean 

Chief  of  Naval  Operations  (OP-115) 

Washington,  DC  20350 

'.AV  224-4970 

'.Commercial  (202)  694-4970 


CAPT  Derr,  USN 
Chief  Naval  Operations 
OP  514,  Room  4D371 
Navy  Department 
The  Pentagon 
Washington,  DC  20350 

LT  Michel Ine  Eyraud,  MSC 
US  Naval  Safety  Center 
Code  1212 
Naval  Air  Station 
Norfolk,  VA  23511 
(804)  44-7926 

Mr.  John  Fargher 

Light  Armored  Vehicle  Directorate 
Development  Center,  MCDEC 
Ouantlco,  VA  22134 
'.AV  278-2092/5 
Commercial  (703)  640-2092/5 

Mr.  Gerald  J.  Fox 
Head,  Life  Sciences 
M.S.  B25-35 

Grumman  Aerospace  Corp. 

Bethpage,  NY  11714 

LCDR  Larry  Frank 
Code  W— 71 2 

Naval  Training  Equipment  Center 
Orlando,  FL  32813 
'.Commercial  (305  )  646-5130 

Mr.  Clarence  Fry 

U.S.  Army  Human  Eng.  Lab. 

ATTN:  DRXHE-AD 

Aberdeen  Proving  Ground,  MD  21005 
'.Commercial  (301)  278-2822 

CDR  Joseph  Funaro 

Naval  Air  Development  Center 

Code  602 

Warminster,  PA  18974 
'.AV  441-2411 

'.Commercial  (215)  441-2411 

Dr.  Richard  F.  Gabriel 
Douglas  Aircraft  Co. 

C1-E80,  Code  35-36 
3855  Lakewood  Blvd. 

Long  Beach,  CA  90846 
'.Commercial  (213)  593-8642 


CAPT.  Thomas  Gallagher 
Code  703 

Naval  Air  Development  Center 
Warminster,  PA  18974 
'.AV  441-2191 

'.Commercial  (215)  441-2191 

CDR  Richard  Gibson,  Head 
Aerospace  Psychology  Office  (Med  3C13) 
Bureau  of  Medicine  and  Surgery 
Naval  Department 
Washington,  DC  20372 

Dr.  Floyd  Glenn 

Analytics 

2500  Maryland  Rd. 

Willow  Grove,  PA  19090 
Commercial  (215)  657-4100 

CDR  David  M.  Graves 
ODASN  (M) 

The  Pentagon  RM  5D800 
Washington,  DC  20350 
'.AV  227-2427 

'.Commercial  (202)  697-2427 

Dr.  Douglas  Harris 

President  and  Principal  Scientist 

Anacapa  Sciences,  Inc. 

P.0.  Drawer  Q 

Santa  Barbara,  CA  93102 

Ms.  Sandra  Hart 
NASA  Ames  Research  Center 
ATTN:  MS  239-3 
Moffett  Field,  CA  94035 
'.Commercial  (415)  965-6072 

Ms.  Christine  Hartel 

U.S.  Army  Research  Institute 

PERI -SM  (Hartel) 

5001  Elsenhower  Ave. 

Alexandria,  VA  22333 

Mr.  John  F.  Hayes 
US  Army  Research  Institute 
ATTN:  PERI -I I  (Mr.  Hayes) 

5001  Elsenhower  Ave. 

Alexandria,  VA  22333 
Commercial  (202)  274-8695 
Autovon  284-8695 


LCOn  Wade  Helm 
Code  6021 

Naval  Air  Development  Center 
Warminster,  PA  18974 
'.AV  441-2561 

'.Commercial  (215)  441-2561 

Ms.  Laura  Hitchcock 
Essex  Corporation,  Suite  200 
65  West  Street  Road 
Warminster,  PA  18974 
Commercial  (215)  674-8710 

Dr.  Lloyd  Hitchcock 

Federal  Aviation  Administration 

ACT-230 

Atlantic  City,  NJ  08405 
'.Commercial  (609)  641-8200  X3378/3379 

Dr.  Richard  Hornlck 
Head,  HF  A  Systems  Safety 
Systems  Division 
Hughes  Ground  Systems 
Bldg.  606,  MS  K214 
Fullerton,  CA  92634 

Mr.  Wayne  Hostetter 
Allen  Corp.  of  America 
3751  Maguire  Blvd. 

Suite  270 
Orlando,  FL  32803 
Commercial  (305)  896-9300 

nr.  Shang  Hwa  Hsu 
XYZYX  Information  Corp. 

21116  Vanowen  Street 
Canoga  Park,  CA  91303 
Commercial  (213)  883-8200 

CDR  Charles  W.  Hutchins 
Naval  Postgraduate  School 
Code  55 

Monterey,  CA  93940 

Dr.  Kay  Inaba 

XYZYX  Information  Corp. 

21116  Vanowen  Street 
Canoga  Park,  CA  91303 
Commercial  (213)  883-8200 


Dr.  Edgar  M.  Johnson 
U.S.  Army  Research  Institute 
5001  Elsenhower  Ave. 
Alexandria,  VA  22333 
'.AV  284-8921 

'.Commercial  (703)  274-8921 

Mr.  Jimmie  H.  Johnson 
Essex  Corp. 

333  N.  Fairfax  St. 

Alexandria,  VA  22314 

Dr.  Richard  Johnson 
U.S.  Army  Research  Institute 
5001  Elsenhower  Avenue 
Alexandria,  VA  22333 
'.AV  284-8943 

'.Commercial  (202)  274-8943 

CDR  Tom  Jones 

Naval  Air  Systems  Command 

Code  340F 

Washington,  DC  20361 
'.(202)  692-7418/9 

Mr.  Paul  Kaster 
Eagle  Technology,  Inc. 

2300  South  9th  St.,  Suite  400 
Arlington,  VA  22204 
'.Commercial  (703)  979-8300 

Dr.  Milton  Katz 
U.S.  Army  Institute 
5001  Elsenhower  Ave. 
Alexandria,  VA  22333 
'.Commercial  (703)  274-8622 

Dr.  Robin  L.  Keesee 

U.S.  Army  Research  Institute 

ATTN:  PERI-SM 

5001  Elsenhower  Ave. 

Alexandria,  VA  22333 

'.AV  284-8905 

'.(202)  274-8905 

Dr.  R.  Kennedy 

Canyon  Research/Suite  227 

1040  Woodcock  Rd. 

Orlando,  FL  32803 
'.Commercial  (305)  894-5090 


Mr.  Harland  K1  dwell 

Singer  Link  Flight  Simulation  Division 

2224  Bay  Area  Blvd. 

Houston,  TX  77058 
Commercial  (713)  488-5510 

Dr.  Judy  Konacki 
EES/EDC 

Georgia  Institute  of  Tech. 

O'Keefe  Blvd. 

Atlanta,  GA  30332 
'.Commercial  (404)  894-3844 

Mr.  Larry  C.  Lamb 

Harris  Corporation 

Gov't  Electronics  Systems  Dlv. 

P.0.  Box  37 
Melbourne,  FL  32901 
Commercial  (305)  729-7127 

Dr.  Joseph  V.  Lambert 
6340  East  Valley  Green  Rd. 

Flourtown,  PA  19031 
'.Commercial  (215)  233-5671 

CDR  Norman  Lane 

Naval  Training  Equipment  Center 
Code  N-7A 
Orlando,  FL  32813 
'.Commercial  (305)  646-5317/5964 

Mr.  William  Lane 

Naval  Training  Equipment  Center 

Code  N— 71 

Orlando,  FL  32813 

'.AV  791-5692 

'.Commercial  (305)  646-5692 

Mr.  Warren  Lewis 
Code  8231 

Naval  Ocean  Sys.  Center 
San  Diego,  CA  92152 
' AV  933-737? 

^Commercial  (714)  225-7372 

LT  Michael  Lllenthal 
Pacific  Missile  Test  Center 
Code  1226 

Point  Mugu,  CA  93042 


Mr.  Paul  Linton 

Naval  Air  Development  Center 

Code  6021 

Warminster,  PA  18974 
'.AV  441-2561 

'.Commercial  (215)  441-2561 

Ms.  Linda  Lund 
Lund  Consulting 
1903  E.  Main  St. 

Box  315 

Mohegan  Lake,  NY  10547 
'.Commercial  (914)  528-8709 

LT  Dennis  McBride  (51) 

Chief,  Human  Factors  Psych.  Div. 
Naval  Aerospace  Med.  Rsch.  Lab. 
NAS  Pensacola,  FL  32507 

Dr.  L.  Bruce  McDonald 
McDonald  &  Associates,  Inc. 

988  Woodcock  Rd. 

Suite  136 
Orlando,  FL  32803 
Commercial  (305)  896-8539 

Dr.  James  McGrath 
NPRDC ,  Code  302 
San  Diego,  CA  92152 
'.AV  933-6617 

'.Commercial  (714)  225-6617 

Mr.  Dale  E.  Mahar 
Pacific  Missile  Test  Center 
Human  Factors  Engineering  Branch 
Code  1226 

Point  Mugu,  CA  93042 
Commercial  (805)  982-8981 
Autovon  351-8981 

Mr.  Gerald  S.  Maleckl 
Code  442  EP 

Office  of  Naval  Research 
800  N.  Quincy  Street 
Arlington,  V A  22217 
Commercial  (202)  696-4741 
Autovon  226-4741 


RAW  R.  Martini,  USN 
Chief  Naval  Operations 
OP51 -Room  4E360 
Navy  Department 
The  Pentagon 
Washington,  DC  20350 

Dr.  David  Melster 
Navy  Personnel  RSD  Center 
San  Diego,  CA  92152 
'.Commercial  (714)  225-6617 

Mr.  Steve  Merrlman 
Code  6022 

Naval  Air  Development  Center 
Warminster,  PA  18974 
'.AV  441-2411/2891 
'.Commercial  (215)  441-2411/2891 

Mr.  J.  Miller 
Life  Sciences 
P.0.  Box  516 
McDonnell -Douglas 
Astronautics  Co. 

St.  Louis,  MO  63166 

LCDR  T.  Mitchell 

Pacific  Missile  Test  Center 

Code  1226 

Point  Mugu,  CA  93042 
'.Commercial  (805  )  982-8981 

Dr.  John  B.  Mocharnuk 

Harris  Corporation 

Government  Electronics  Systems  Dlv. 

P.0.  Box  37 

Melbourne,  FL  32901 

Commercial  (305)  727-6131 

Dr.  Tony  Modrick 
Honeywell,  Inc. 

MN  17-2314 
2600  Rldgway  Parkway 
Minneapolis,  MN  55440 
Commercial  (612)  378-5016 

Dr.  Heber  G.  Moore 
HDQ  Naval  Material  Command 
ATTN:  MAT  08D4B 
Washington,  DC  20360 
'.Commercial  (202)  692-2646 


Mr.  Ross  Morgan 
Technical  Director 
HRL/LR 

Wrlght-Patterson  AFB,  OH  45433 

CDR  William  F.  Moroney 
SY  72 

Naval  Air  Test  Center 
Patuxent  River,  MD  20670 

Dr.  Fred  Muckier 

Canyon  Research  Group,  Inc. 

2135  Hartford  St. 

San  Diego,  CA  92110 
'.Commercial  (213)  889-5072 

Mr.  W.  G.  Muller 
Systems  4  Research  Center 
PO  Box  312 
2600  Ridgway  Parkway 
Stop  MN  172307 
Minneapolis,  MN  55440 
'.Commercial  (612)  378-5086 

Mr.  Miles  R.  Murphy 
Aviation  Safety  Research  Office 
NASA-Anes  Research  Center 
M.S.  239-2 

Moffett  Field,  CA  94035 
'.Commercial  (415)  965-5906 

Mrs.  Loulda  D.  Murray 
Eagle  Technology,  Inc. 

320  West  Street  Road 
Warminster,  PA  18974 
Commercial  (215)  672-6250 

LCDR  Kevin  Myette 
Chief  Naval  Air  Forces 
Atlantic  Staff 
NAS 

Norfolk,  YA  23502 

Mr.  Gregory  J.  Opetosky 
Lund  Consulting,  Inc. 

1903  E.  Main  St. 

P.0.  Box  315 

Mohegan  Lake,  NY  10547 

Commercial  (914)  528-8709 


Or.  Jesse  Orlansky 
Institute  for  Defense  Analyses 
1801  N.  Beauregard  Street 
Alexandria,  VA  22311 
Commercial  (703)  845-2293 


Mr.  Curt  Sandler 
NAVAIRSYSCOM 
AIR-531 3D 

Washington,  DC  20361 
Commercial  (202)  692-7486/7 


LCDR  Jerry  Owens 

Naval  Aerospace  Medical  Research  Lab. 
Pensacola,  FL  32508 

Mr.  William  J.  Phillips 
Computer  Sciences  Corporation 
6565  Arlington  Blvd. 

Falls  Church,  VA  22046 
Commercial  (703)  237-2000 

LCDR  M.  J.  Pianka 
Code  SY  50 
SETD 

Naval  Air  Test  Center 
Patuxent  River,  MD  20670 
Commercial  (301)  863-4146 
Autovon  356-4146 

Mr.  Joseph  Pontecorvo 
Federal  Aviation  Administration 
Aircraft  Maintenance  Division 
800  Indepedence  Avenue,  SW 
Washington,  DC  20591 
Commercial  (202)  426-3546 
Autovon  426-3546  (FTS) 

Mr.  William  Rizzo 

Naval  Training  Equipment  Center 

Code  N— 71 2 

Orlando,  FL  32813 

Commercial  (305)  646-5130 

Autovon  791-5130 

Dr.  William  B.  Rouse 

Center  For  Man-Machine  Systems  Research 

Georgia  Institute  of  Technology 

Atlanta,  GA  30332 

Commercial  (404)  894-3996 

Mr.  Arnold  Rubensteln 
Code  NAVMAT 
08D/07255 
Room  1044 

Naval  Material  Command 
Crystal  Plaza  5 
Washington,  DC  20360 


Dr.  Richard  Schlffler 
ASD/ENECH 

Wrlght-Patterson  AFB,  OH  45433 
'.AV  785-5597 

'.Commercial  (513)  255-5597 

Dr.  Sam  Schlflett 
Aircrew  Systems 
SY  70E 

Naval  Air  Test  Center 
Patuxent  River,  MC  20670 
'.AV  356-4157 

'.Commercial  (301)  863-4157 

Dr.  John  C.  Schmitt 

Harris  Corporation 

Gov't  Electronics  Systems  Div. 

P.0.  Box  37 

Melbourne,  FL  32901 

Commercial  (305)  729-7T27 

Dr.  Robert  Smith,  Jr. 

Office  of  the  Chief  of  Naval 
Operations 
Code  0P-987H 
The  Pentagon 
Washington,  DC  20350 
'.AV  227-1219 

'.Commercial  (202)  697-1219 

Mr.  Joseph  Stahl 

Human  Factors  Project  Engineer 

Ballistic  Missile  Division 

TRW  Systems,  Bldg.  526/620 

P.0.  Box  1310 

San  Bernardino,  CA  92402 

Dr.  C.  J.  Thelsen,  Jr. 

Essex  Corporation 
65  West  Street  Road 
Suite  C-200 
Warminster,  PA  18974 
Commercial  (215)  674-8710 


Hr.  Guice  Tinsley 

Federal  Aviation  Administration 

flOO  Independence  Ave. 

Washington,  DC  20591 
Commercial  (202)  426-9472 

Dr.  Martin  A.  Tolcott 
Office  of  Naval  Resrch. 

800  N.  Quincy  St. 

Arlington,  VA  22217 

Dr.  Donald  Topmlller 
AFAMRL/HED 

Wrl ght-Patterson  AFB,  OH  45433 

Dr.  Douglas  M.  Towne 

Behavioral  Technology  Laboratories 

1845  South  Elena  Avenue 

Fourth  Floor 

Redondo  Beach,  CA  90277 

Commercial  (213)  540-3654 

Mr.  Martin  A.  Traub 
Appll-Matlon,  Inc. 

6955  Hanging  Moss  Rd. 

Orlando,  FL  32807 
Commercial  (305)  678-6955 

Mr.  Marshall  Whlteaker 
HO  AFLC/LOEP 

Wrl ght-Patterson  AFB,  OH  45433 

Mr.  Edward  R.  Winkler 
McDonnell  Douglas  Corporation 
Dept  E423,  Bldg  271  C/1 /MSI 68 
P.0.  Box  516 
St.  Louis,  MO  63166 
Commercial  (314)  233-7096 

Mr.  T.  J.  Wong 
Douglas  Aircraft  Co. 

MS  35-36 

3855  Lakewood  Blvd. 

Long  Beach,  CA  90846 
(213)  593-8641 
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